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PREFACE 


To  those  who  have  grappled,  seriously  with  important  prob- 
lems of  choice  in  their  personal  spheres  (such  as  purchasing  a 
home,  deciding  which  automobile  to  buy,  choosing  among  job 
options,  etc.)  the  difficulties  inherent  in  decision  making  are 
apparent.  Even  at  the  relatively  uncomplicated  level  of  per- 
sonal choice,  we  often  encounter  more  relevant  decision  dimen- 
sions than  the  intellect  can  cope  with,  many  dimensions  are 
difficult  to  value  in  an  objective  way,  and  a highly  uncertain 
world  is  usually  interposed  between  possible  choices  and  their 
outcomes . 


While  moat  of  us  cope  with  personal  decisions  with  more  or 
less  systematic  consideration  (and  usually  with  unknown  degrees 
of  success  or  failure) , it  takes  little  extrapolation  to  realize 
that  in  national  security  decision  contexts,  the  problems  are 
often  far  more  complex,  the  uncertainties  greater,  and  the 
stakes  involved  of  enormous  magnitude.  These  factors,  coupled 
with  a research  base  that  points  to  suboptimal  human  perfor- 
mance in  complex  decision  tasks, 1 have  served  as  the  impetus 
for  the  development  of  formal  methodologies  to  aid  decision 
makers.  During  World  War  II,  formal  approaches  to  decision- 
making, especially  in  defense  settings,  began  to  be  introduced 
under  the  name  of  operations  research.  They  were  typically 
applied  to  special  types  of  clearcut,  repetitive  problems,  such 
as  those  of  systematic  search  and  resource . allocation. 

Since  the  1960 's,  however,  a more  general  technology  has 
emerged  for  imposing  logical  structure  on  the  reasoning  that 
underlies  any  specific  decision.  This  technology  is  decision 
analysis.  Since  1970,  there  has  been  a dramatic  burgeoning  of 
efforts  by  defense  agencies  to  adapt  this  technology  to -their 
day-to-day  decision  making.  Many  have  found  it  a way  to  make 
better,  more  defensible  decisions. 

Decision  analysis  is  a quantitative  method  which  permits 
the  systematic  evaluation  of  the  costs  or  benefits  accruing  to 
courses  of  action  that  might  be  taken  in  a decision  problem. 

It  entails  identification  of  the  alternative  choices  involved, 
the  assignment  of  values  (costs/benef its)  for  possible  outcomes. 


For  excellent  reviews  of  this  research  base,  see  Becher,  G.  M. 
and  McClintock,  C.  G. , "Value;  Behavioral  Decision  Theory," 
Annual  Review  of  Psychology  18:239-286  (1967);  Rapoport,  A. 
and  Wallston,  T.  S.,  Individual  Decision  Behavior,"  Annual 
Review  of  Psychology  23:131-175  (1972);  Slovic,  P., 

Fischhof f , B. , and  Lichtenstein,  S.,  "Behavioral  Decision 
Theory,"  Annual  Review  of  Psychology  26:1-39  (1977). 


and  the  expression  of  the  probability  of  those  outcomes  being 
realized.  With  this  information  at  hand,  one  can  then  sys- 
tematically combine  the  values  and  probabilities  to  show  the 
probable  gain  or  loss  that  is  associated  with  alternative 
choices. 

In  the  application  of  decision  analysis,  a problem  is 
decomposed  into  clearly  defined  components  in  which  all  options, 
outcomes,  values,  and  probabilities  are  depicted.  Quantifica- 
tion in  the  form  of  the  value  for  each  possible  outcome  and  the 
probability  of  those  values  (or  costs)  being  realized  can  be  in 
terms  of  objective  information  or  in  the  form  of  quantitative 
expressions  of  the  subjective  judgments  of  experts.  In  the 
latter  case,  the  quantitative  expression  serves  to  make  ex- 
plicit those  subjective  qualities  which  would  otherwise  be 
weighed  in  the  decision  process,  albeit  in  a more  elusive, 
intuitive  way. 

Beyond  its  primary  role  of  serving  as  a method  for  the 
logical  solution  of  complex  decision  problems,  decision  analy- 
sis has  additional  advantages  as  well.  The  formal  structure  of 
decision  analysis  makes  clear  all  the  elements,  their  relation- 
ships, and  their  associated  weights  that  have  been  considered 
in  a decision  problem.  If  only  because  the  model  is  explicit, 
it  can  serve  an  important  role  in  facilitating  communication 
among  those  involved  in  the  decision  process.  With  a decision 
problem  structured  in  a decision  analytic  framework,  it  is  an 
easy  matter  to  identify  the  location,  extent,  and  importance  of 
any  areas  of  disagreement,  and  to  determine  whether  such  dis- 
agreements have  any  material  impact  on  the  indicated  decision. 

In  addition,  should  there  be  any  change  in  the  circumstances 
bearing  upon  a given  decision  problem,  it  is  fairly  straight- 
forward to  reenter  the  existing  problem  structure  to  change 
values  or  to  add  or  remove  problem  dimensions  as  required. 

It  shouJ.d  be  emphasized  that  in  no  sense  does  decision 
analysis  replace  decision  makers  with  arithmetic  or  change  the 
role  of  wise  human  judgment  in  decision  making.  Rather,  it 
provides  an  orderly  and  more  easily  understood  structure  that 
helps  to  aggregate  the  wisdom  of  experts  on  the  many  topics 
that  may  be  needed  to  make  a decision,  and  it  supports  the 
skilled  decision  maker  by  providing  him  with  logically  sound 
techniques  to  support,  supplement,  and  ensure  the  internal 
consistency  of  his  judgments. 

This  handbook  is  intended  to  provide  decision  makers  and 
their  staffs  (current  or  potential)  with  an  introduction  to  the 
basic  concepts  and  operations  of  decision  analysis.  As  an 
introductory  treatment,  it  is  not  expected  that  mastery  of  the 
material  presented  here  will  make  an  expert  decision  analyst  of 
the  reader.  Rather,  it  is  our  hope  that  the  material  will 
provide  readers  a level  of  acquaintance  with  the  methodology 
that  will  onable  them  to  incorporate  at  least  elements  of 
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decision  analysis  into  their  professional  decision  making 
activities,  to  be  generally  aware  of  the  capabilities  of  deci- 
sion analysis,  to  recognize  decision  contexts  meriting  formal 
decision  analysis,  and  to  be  able  to  participate  in  and  eval- 
uate decision  analysis  applications  which  will  assuredly  be 
encountered  with  increasing  frequency. 

Chapters  1 through  5 of  this  handbook  provide  an  overview 
of  the  major  technical  aspects  of  decision  analysis:  struc- 
turing models,  assigning  probabilities  and  values,  concepts  of 
personal  probabilities,  developing  inferences  from  evidence, 
and  information  value.  The  concepts  are  developed  in  the 
context  of  defense  or  defense-related  examples  which  we  hope 
will  enable  our  principal  reading  audience  to  better  relate  to 
the  material.  Chapter  6 provides  a case  study  in  the  form  of  a 
dialogue  between  a decision  analyst  and  a task  force  commander 
centered  around  the  use  of  decision  analysis  in  resolving  a 
decision  problem  faced  by  the  commander.  The  case  study  is 
designed  to  exemplify  all  the  principles  developed  in  Chapters 
1 through  5.  Thus,  it  can  be  used  as  a capstone  for  those  who 
choose  first  to  step  through  the  development  of  the  principles. 
Alternatively,  those  seeking  the  quickest  possible  acquaintance 
with  what  decision  analysis  is  about  and  roughly  how  it  works 
in  practice  may  review  Chapter  6 first.  Cross  references  in 
Chapter  6 to  earlier  sections  of  the  text  which  elaborate  the 
principles  being  applied  in  the  case  study  will  guide  the 
reader  to  fuller  comprehension. 

For  the  reader  who  elects  to  go  into  the  subject  matter  in 
greater  depth,  excellent  texts  exist.  Decision  Analysis  for 
the  Manager  by  R.  V.  Brown,  A.  S.  Kahr,  and  C.  R.  Peterson  TNew 
York:  Holt,  Rinehart,  and  Winston,  1974)  is  an  up-to-date  and 
comprehensive  treatment  of  decision  analysis  and  its  appli- 
cations. Replete  with  examples  of  applications  of  decision 
analysis  in  business  contexts,  the  text  is  fairly  easy  reading 
with  only  very  elementary  mathematics  used.  Howard  Raiffa's 
book.  Decision  Analysis;  Introductory  Lectures  on  Choices 
Under  Uncertainty  (Addison-WesTey,  1968)  was  the  first  compre- 
hensive treatment  of  the  theory  and  application  of  decision 
analysis.  Somewhat  more  mathematical  than  the  Brown,  Kahr, 
Peterson  book,  Raiffa's  text  is  still  an  excellent  introduction 
to  the  subject  matter.  A non-mathematical  treatment  of  deci- 
sion analysis  can  be  found  in  Analysis  of  Decisions  Under 
Uncertainty  by  Robert  Schlaiffer  (McGraw-Hill , 1&6^) . Schlaiffi 
book  develops  all  the  key  ideas  of  decision  analysis  logically 
and  intuitively  without  recourse  to  the  mathematical  under- 
pinnings. A much  shorter  and  readily-comprehensible  treatment 
of  decision  analysis  is  to  be  found  in  Dennis  Bindley 's  book, 
Making  Decisions  (Wiley,  1973) . For  the  student  desiring  a 
varied  motivational  approach  to  the  subject  matter,  Howard 
Raiffa  has  prepared  an  excellent  self-instructional  program 
that  uses  lectures,  discussions,  readings,  examples,  and 
exercises.  Presented  in  the  form  of  audio  cassettes  and 
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supporting  printed  materials,  the  course  entitled  Analysis  for 
Decision  Making;  An  Audio-graphic,  Self-Instructional  Course, 
is  available  from  tHe  Encyclopedia  Britannica  Educational 
Corporation. 
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CHAPTER  1 


DIAGRAMING  AND  SOLVING  DECISION  PROBLEMS 


Complex  decision  problems  can  be  difficult  to  resolve  for 
a variety  of  reasons.  Frequently,  options  are  not  clearly 
d led,  the  results  that  might  be  achieved  by  opting  for  one 
choice  over  another  may  be  highly  uncertain,  and  it  is  often 
difficult  to  determine  relative  preferences  for  the  possible 
decision  outcomes.  Certainly,  almost  everyone  has  encountered 
decision  problems  characterized  by  such  uncertainty.  Usually, 
the  reaction  is  either  to  devote  more  thought  to  the  circum- 
stances than  would  normally  be  afforded,  or  to  resort  to 
various  devices  to  help  sort  out  the  decision  such  as  listing 
pros  and  cons  for  each  option,  rank  ordering  preferences, 
listing  the  things  that  could  go  wrong,  and  so  on.  In  either 
case,  whether  through  extended  contemplation  of  the  problem  or 
through  recourse  to  more  explicit  written  aids,  the  person  with 
the  problem,  the  decision  maker,  attempts  to  lend  structure  to 
the  problem  to  reduce  it  to  a more  explicit,  tractable  form. 

In  a much  more  systematic  and  formal  way,  that  is  exactly  what 
decision  analysis  helps  him  do. 

Although  there  are  many  alternative  ways  to  structure 
decision  problems,  decision  analysis  build's  upon  four  basic 
elements  that  are  inherent  in  any  decision  problem.  These 
elements  lead  to  a rather  natural  way  to  conceptualize  and 
resolve  complex  decisions. 

The  first  element  is  a set  of  initial  courses  of  action. 
You  have  a decision  to  make  only  if  you  face  a choice  among 
alternative  possible  acts.  Each  of  the  choices  you  want  to 
consider  should  be  made  explicit. 

Second,  one  needs  to  consider  the  possible  consequences  of 
each  initial  act.  What  are  the  important  things  that  can 
happen  that  will  make  one  act  more  valuable  or  worth  more  than 
another  act?  Relevant  sequences  of  subsequent  events  and 
follow-up  acts  must  be  identified  for  each  initial  act. 

The  third  element  is  concerned  with  how  attractive  or 
unattractive  each  possible  consequence  of  each  act  is  to  you. 
What  is  its  value  to  you?  How  undesirable  is  one  outcome 
compared  to  others  which  might  result  from  the  same  or  another 
decision?  This  value  could  be  measured  in  terms  of  money, 
utility,  or  some  other  carefully  defined  index. 

Finally,  how  likely  is  it  that  a particular  act  will 
result  in  each  o£  the  consequences?  This  uncertainty  may  be 
measured  either  by  a numerical  probability  from  0 to  1 or  in 
the  form  of  odds. 
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Decision-analytic  methodology  provides  a way  to  quantify, 
organize,  and  trace  the  logical  implications  of  the  decision 
elements  defined  above.  The  primary  object  of  decision 
analysis  is  to  model  the  decision,  or  some  part  of  it,  so  that 
at  least  some  of  the  implications  can  be  deduced.  By  the  verb 
"model,"  we  mean  to  represent  in  a quantified  form.  The 
resulting  model  may  be  complete  in  the  sense  that  it  incor- 
porates all  of  the  decision  maker's  relevant  perceptions  of 
alternatives  of  value  or  uncertainty.  In  that  case,  if  it  is 
an  accurate  model,  it  will  determine  exactly  what  course  of 
action  should  be  taken. 

On  the  other  hand,  it  may  be,  and  usually  is,  the  case 
that  it  is  a partial  model;  it  displays  the  logical  implica- 
tions of  some,  but  not  necessarily  all,  of  the  relevant  inputs. 
It  may,  for  example,  simply  display  the  probable  monetary 
consequences  for  selected  choices,  leaving  non-monetary  con- 
sequences and  other  choices  for  later  consideration.  In  that 
case,  it  is  not  necessary  for  the  initial  course  of  action 
that  is  favored  by  the  model  to  be  the  one  that  is  decided 
upon.  For  example,  it  may  be  found  that  the  monetary  conse- 
quences that  favor  one  course  of  action  are  far  outweighed  by 
the  non-monetary  criteria  that  favor  another  action. 

For  the  present  we  shall  focus  on  complete  decision 
analytic  models  because  they  are  clearest  for  purposes  of 
exposition.  Later,  we  consider  some  of  the  shortcuts  that  are 
necessary  to  reduce  the  task  of  a decision  analysis  to  manage- 
able size. 

The  Structure  of  Decision  Problems;  Decision  Diagrams 

It  is  a central  tenet  of  decision  analysis  that  all 
relevant  considerations  in  a decision  can  be  assigned  to  one 
or  another  of  the  four  components:  initial  options,  possible 
consequences,  values,  and  uncertainties.  In  addition,  they 
can,  in  principle,  be  represented  fully  in  a decision  diagram. 
In  other  words,  for  every  conceivable  decision,  it  is  theo- 
retically possible  to  construct  a decision  diagram  which 
captures  everything  a decision  maker  feels  is  relevant  to  the 
choice  in  question. 

A decision  diagram  is  a kind  of  road  map.  Like  a road 
map,  it  visually  displays  possible  destinations  (outcomes)  at 
which  you  may  arrive  if  you  take  one  or  another  of  the  routes 
(initial  acts)  immediately  available  to  you.  It  also  shows 
you  what  you  may  come  across  on  the  way  (events)  and  what 
later  choices  (subsequent  acts)  you  may  liave  to  make. 

A decision  diagram  consists  essentially  of  a network  of 
branches  corresponding  to  possible  sequences  of  acts  and 
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events , fanning  out  from  an  origin  at  the  left  to  a time 
Horizon  at  the  right.  Acts  are  available  choices.  Events  are 
possible  occurrences  which  are  partly  or  completely  outside 
the  decision  maker's  direct  control,  though  the  chance  of  one 
of  them  happening  may  be  influenced  by  acta  that  were  carried 
out  earlier.  The  diagram  is  made  up  of  a concatenation  of 
forks  which  are  either  act  forks  or  event  forks.  A path 
through  the  diagram  corresponds  to  a possible  sequence  of  act.s 
and  events  characterized  by  a value  assigned  to  the  conse- 
quence (but  we  will  deal  with  matters  of  value  later) . 

The  decision  diagram  graphically  distinguishes  acta  from 
events.  Act  forks  are  represented  by  squares,  and  event  forks 
are  represented  by  circles,  as  shown  in  Figure  1-1. 


ACT  FORK 


OPTION  A 


EVENT  FORK 

OCCURRENCE  X 


Figure  1-1 

REPRESENTATIONS  OF  ACT  AND  EVENT  FORKS 


Although  many  examples  throughout  this  text  will  alternate  act 
forks  and  event  forks,  it  is  not  necessary  that  acts  and 
events  alternate  in  sequence.  By  definition,  however,  the 
decision  diagram  must  begin  with  an  act  fork  at  the  point  of 
origin  at  the  left  and  end  with  an  event  circle  at  the  ter- 
mination of  each  branch  at  the  time  horizon  on  the  right. 
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To  see  how  a decision  diagram  is  developed,  let's  con- 
sider a simple  example.  Suppose  as  a defense  contractor,  you 
must  decide  whether  or  not  to  bid  on  Contract  A.  To  represent 
this  situation,  an  initial  act  fork  is  required: 


BID  ON  A 


Figure  1-2 

INITIAL  ACT  FORK 


The  possible  outcomes  of  bidding  on  A are  winning  the  bid  and 
losing  it.  To  represent  this  situation,  an  event  fork  is 
drawn  at  the  right  end  of  the  bid-on-A  line: 


WIN  A 


Figure  13 

ACT  A WITH  EVENT  FORK  ADDED 


At  this  point,  the  decision  diagram  is  rather  small  and 
simple,  as  shown  in  Figure  1-4. 

Suppose  further  that,  after  deciding  whether  or  not  to 
bid  on  A,  you  must  decide  whether  or  not  to  bid  on  Contract  B. 
This  situation  is  represented  by  an  act  fork  like  the  initial 
act  fork  in  Figure  1-2.  However,  since  a bid  on  B is  inde- 
pendent of  your  initial  decision,  this  act  fork  is  equally 
applicable  to  three  situations:  bidding  on  A and  winning, 
bidding  on  A and  losing,  and  not  bidding  on  A at  all.  To 
represent  this  situation,  a larger  and  more  complex  decision 
diagram  is  required,  as  shown  in  Figure  1-5.  Needless  to  say, 
the  possible  outcomes  of  bidding  on  B are  winning  the  bid  and 
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Figure  1-4 

STRUCTURE  OF  BIDDING  DECISION 


losing  It,  which  outcomes  mny  be  represented  by  an  event  fork 
analagouB  to  that  shown  In  Figure  1-3. 

Clearly,  we  can  continue  this  process  of  extending  the 
decision  diagram;  we  can  add  more  contracts  or  vary  the  event 
forks  of  possible  consequences  of  winning  or  losing  additional 
contracts.  Note  that  the  branch  representing  a bid  on  A 
alternates  act  forks  and  event  forks,  but  that  the  branch 
representing  no  bid  on  A does  not  alternate  act  and  event 
forks.  We  do  not  show  an  event  fork  following  an  event  fork, 
but  It  is  easy  to  Imagine  events  like  merger,  bankruptcy,  and 
the  like  following  the  outcome  of  winning  or  losing  a bid  on  B. 


Figure  1-5 

STRUCTURE  OF  BIDDING  DECISION 
WITH  SECOND  ACT  FORK  REPRESENTED 
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The  decision  diagram  in  Figure  1-6  reflects  the  structure 
of  the  bidding  situation  we  originally  described,  in  which  the 
coiitracts  were  unrelated.  But  the  technique  of  constructing 
decision  diagrams  is  flexible  enough  to  cover  other,  more 


WIN  B 


Figure  1-6 

BIDDING  DECISION  STRUCTURE  FOR  INDEPENDENT  BIDS 


complex  situations.  Suppose,  for  example,  that  two  different 
government  policies  establish  different  contract  situations, 
especially  in  terms  of  the  relationship  between  bids  on  con- 
tracts A and  B.  On  the  one  hand,  it  may  be  stipulated  that 
whoever  wins  A must  bid  on  B.  The  stipulation  may  be  based  on 
the  belief  that  the  winner  of  A is  in  the  best  position  to  do 
the  work  required  by  B.  This  situation  is  represented  in  the 
decision  diagram  shown  in  Figure  1-7.  Alternatively-  it  may 
be  specified  that  whoever  wins  contract  A may  not  bid  on 
contract  D,  a stipulation  based  on  a policy  of  distributing 
government  contracts  throughout  an  industry.  This  modified 
decision  structure  is  shown  in  Figure  1-8. 

In  developing  the  decision  diagrams  for  the  situations 
represented  in  Figures  1-6  through  1-8,  we  followed  two  rules 
governing  the  construction  of  the  diagrams.  The  two  rules 
are: 

RULE  # 1:  ACTS  APPEAR  ON  A DECISION  DIAGRAM  AT  THE 

MOMENT  WHEN  THE  DECISION  MAKER  MUST  IRREVOCABLY 
SELECT  ONE  OPTION  AMONG  SEVERAL  OR  COMMIT 
RESOURCES;  and 
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Rule  1 directs  us  to  represent  actions  in  the  decision  making 
process  when  they  are,  in  fact,  made,  not  when  they  are  con- 
templated. Rule  2 directs  us  to  represent  those  events  in  the 
decision-making  process  when  they  becdme  known  to  the  decision 
maker.  For  only  the  decision-maker's  knowledge  of  events,  not 
the  prior  occurrence  of  the  events  themselves,  can  affect  his 
decision-making  process. 

Thus  far,  the  decision  diagrams  we  have  considered  have 
depicted  only  the  possible  acts  and  events  and  shown  how  these 
relate  to  each  other  in  a decision  situation.  Put  another 
way,  they  have  shown  the  structure  of  the  sample  decision 
problems.  While  the  representations  would  help  a decision 
maker  to  see  at  a glance  his  alternatives  and  identify  those 
things  that  might  affect  any  choice  to  be  made,  tiiey  do  not 
yet  address  the  central  question:  which  choice  should  be 
made?  That  question  cannot  be  answered  from  these  skeletal 
diagrams  since  important  information  about  the  value  of  the 
possible  outcomes  and  the  likelihood  of  occurrence  of  events 
has  not  yet  been  considered.  Once  this  information  is  incor- 
porated into  the  decision  diagram,  a matter  we  turn  to  in  the 
next  section,  we  shall  have  completely  represented  the  essen- 
tial elements  of  the  decision. 

Before  moving  on,  we  should  emphasize  a number  of  points 
about  the  discussion  thus  far.  First,  it  should  be  understood 
that  for  purposes  of  exposition  we  have  used  very  simple 
examples.  Real-world  decisions  of  a size  and  complexity 
meriting  analytic  attention  would  be  much  more  complex  than 
the  simple  examples  we  have  used.  Since  the  diagrams  of  our 
simple  examples  already  extend  across  a page,  the  thoughtful 
reader  may  well  wonder  about  the  tractability  of  more  complex 
problems.  It  is  enough  to  say  at  this  point  that  such  concern 
is  appropriate.  Fortunately,  there  are  shortcuts  that  reduce 
very  complex  decision  problems  to  manageable  proportions.  A 
second  point  to  bear  in  mind  is  that  the  process  of  laying  out 
the  structure  of  a decision  problem  (diagraming  the  problem) 
is  the  most  important  and  perhaps  most  difficult  step  in 
decision  analysis.  If  the  structure  does  not  accurately  rep- 
resent the  factors  bearing  on  a decision,  the  resolution  of 
that  decision  problem  will  be  degraded. 

Representing  Values  and  Probabilities  in  Decision  Diagrams 

As  was  pointed  out  in  the  previous  section,  there  are 
four  essential  elements  in  a decision:  Initial  options, 
possible  consequences,  values,  and  uncertainties.  In  our 
discussion  thus  far,  we  have  covered  the  way  decisions  are 
structured  to  depict  two  of  the  elements:  initial  options 
and  the  possible  consequence.B  of  each  initial  option.  Now  we 
shall  turn  to  the  remaining  elements,  values  and  uncertainties. 
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and  show  how  these  are  incorporated  into  our  decision  repre- 
sentation. For  convenience  we  will  use  the  b iding  rexample, 
variants  of  which  we  diagramed  in  the  preceding  section. 

In  considering  the  decision  of  whether  or  not  to  bid  on 
contract  A and  subsequently  to  bid  on  contract  B«  let  us 
assume  that  there  are  a number  of  relevant  cost  and  value 
dimensions  that  can  be  calculated  or  estimated  to  help  with 
the  decision.  Let's  say  you  have  calculated  that  it  will  cost 
$10,000  to  prepare  a bid  on  contract  A and  that  if  you  win 
that  bid,  you  will  gain  $50,000,  Suppose  further  that  to  bid 
on  contract  B will  cost  $10,000,  just  as  the  bid  on  A did,  but 
that  since,  if  you  win  B after  winning  A,  producing  for  both 
contracts  will  stretch  your  capacity  to  the  point  where  you 
incur  excessive  costs,  your  profit  on  B alone  would  be  only 
$20,000.  Tracing  through  the  uppermost  path  of  the  decision 
diagram  shown  in  Figure  1-9,  you  can  see  these  values  re- 
flected as  follows:  bid  on  A,  -10;  win  A,  -fSO}  bid  on  B,  -10; 
win  B,  ' ^0.  If  you  do  not  bid  on  A,  but  do  on  B,  your  costs 
will  be  riiodest,  leading  to  a $50,000  profit  on  B.  The  sequence 
of  valuer  is:  no  bid  on  A,  0;  bid  on  B,  -10;  win  B,  +50. 
Corresponding  values  are  sho%m  on  the  other  branches  of  the 
diagram  as  well.  The  diagram  now  reflects  three  of  the  ele- 
ments essential  to  decision;  initial  options,  possible  conse- 
quences, and  values. 

The  numbers  shown  at  the  right-hand  side  of  the  diagram 
are  the  path  values.  For  any  specific  path  through  the  dia- 
gram, the  values  and  costs  Involved  in  achieving  the  indicated 
outcomes,  if  they  occvir  for  certain,  will  result  in  the  path 
value  shown.  Tracing  through  the  uppermost  path  of  the  dia- 
gram, we  add  or  subtract  values  shown  below  each  branch  to 
obtain  the  path  value? 

-10 [bid  on  A cost]  + 50 [win  A]  -10 [bid  on  B cost]  + 20 [win  B] 
-50. 

The  values  for  the  remaining  paths  are  calculated  in  a 
directly  comparable  manner.  Note  that  in  calculating  these 
values,  the  probabilities  of  achieving  the  various  outcomes 
did  not  enter  into  the  calculation.  Those  will  come  into  play 
in  the  next  section  when  we  discuss  how  to  solve  a decision 
diagram  to  determine  the  best  course  of  action. 

To  complete  the  representation  of  the  decision  problem  we 
need  to  reflect  in  the  diagram  the  degree  of  uncertainty  about 
the  events  which  we  cannot  control,  that  you  will  win  either 
or  both  of  those  contracts  if  you  should  elect  to  bid  on  them. 
From  prior  experience  or  perhaps  from  an  intangible  "sense  of 
the  situation,"  you  judge  that  you  are  equally  likely  to  win 
or  lose  the  bid  on  contract  A (probability  .£  win,  .5  lose). 
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PATH 
VALUE 
(in  $000) 


WIN  B 


+60 

+30 

+40 


+30 


-20 


-10 


+40 


-10 


0 


If  you  win  the  bid  on  A,  you  judge  your  chances  with  respect 
to  contract  B as  .4  win  and  .6  lose.  On  the  other  hand,  if 
you  lose  the  bid  on  contract  A,  you  assess  your  probability  of 
winning  a bid  on  B to  be  .7  and  the  probability  of  losing  the 
bid  on  B to  be  .3.  These  probability  numbers  are  shown  entered 
in  our  bidding  diagram  in  Figure  1-10.  Tracing  through  th?' 

"Bid  on  A"  path  of  the  diagram,  you  can  see  how  these  uncer- 
tainties (probabilities)  are  represented. 


10 


PATH 
VALUE 
(in  $000) 


BID  ON  B 
-10 


WIN  A 
+60 


WIN  B 


L OSE  B 


NO  BID 


BID  ON  A 
-10 


BID  ON  B 
-10 


LOSE  A 


WIN  B 
•V  +50 


.3\  LOSE  B _20 


BID  ON  B 
-10 


NO  BID 


NO  BID 


WIN  B 
+60 

LOSE  B 


NO  BID 


Figure  1-10 

TREE  FOR  BIDDING  EXAMPLE 


Solving  a Decision  Diagram 


We  have  seen  how  decision  diagrams  are  constructed  and 
shown  how  information  relevant  to  the  decision  is  represented 
in  the  diagram.  In  principle,  such  a diagram  can  be  a vir- 
tually complete  translation  or  model  of  your  perception  of  the 
decision  problem.  If  it  is,  you  can  be  satisfied  that  the 
"best"  decision  in  the  problem  model  is  also  the  best  decision 
in  your  real  world  situation.  Then  you  have  only  to  determine 
which  is  the  "best"  decision.  How  is  this  determination  to  be 
made;  that  is,  which  immediate  option  is  the  most  attractive? 
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The  value  of  outcomes  has  been  previously  introduced  as  a 
measure  of  the  attractiveness  of  a possible  outcome  of  a 
decision  option  when  that  outcome  is  treated  as  certain.  If 
all  outcomes  in  decision  situations  were  in  fact  certain,  we 
would  need  go  no  further  to  solve  our  decision  problem.  We 
would  need  only  to  diagram  the  problem,  calculate  the  value  of 
each  outcome  and  settle  upon  the  choice  that  leads  to  the 
outcome  with  the  highest  value  (or  lowest  cost) . When  uncer- 
tainty enters  the  picture,  however,  as  we  have  noted  in  the 
event  nodes  of  our  bidding  example  in  Figure  1-10,  the  value 
of  an  outcome  is  seldom  the  sole  criterion  for  choosing  among 
decision  options.  You  know  without  thinking,  for  example, 
that  you  would  not  decide  in  favor  of  a gamble  that  offered 
you  one  chance  in  100  of  winning  $10.00  over  one  offering  you 
a 90%  chance  of  winning  $9.00.  In  these  hypothetical  gambles, 
the  first,  winning  $10.0,0,  has  a higher  outcome  value  than  the 
second,  winning  $9.00.  You  intuitively  would  take  the  second 
choice,  the  lower  outcome  value,  out  of  a sense  of  the  like- 
lihood of  winning.  What  you  have  done  implicitly  is  to  weight 
the  outcome  values  by  the  probability  of  achieving  that  value. 
As  we  shall  see  below,  the  solution  of  decision  diagrams 
involves  determining  weighted  values  that  accrue  to  alterna- 
tive acts  (decision  options)  In  a decision  problem.  There  are 
a variety  of  ways  of  making  these  determinations.  In  the 
remainder  of  this  chapter  we  shell  present  but  one  method  of 
solving  decision  diagrams.  The  method  involves  the  calcula- 
tion of  weighted  values,  often  called  expected  values,  for 
each  decision  option.  This  technique  is  the  simplest  and 
statistically  most  straightforward  method  that  can  be  used  and 
is  thus  useful  for  demonitratxng  the  general  principles  in- 
volved in  solving  a decision  problem.  The  reader  is  cautioned, 
however,  that  frequently  there  are  decision  circumstances 
wherein  an  expected  value  solution  to  a decision  problem  may 
not  be  an  optimal  one.  In  these  cases,  alternative  means  of 
treating  value  are  required.  These  alternative  methods  are 
covered  in  Chapter  2. 

Inferring  Act  Values  By  Folding  Back  a Decision  Diagram 

Determining  a value  for  each  immediate  act  in  a decision 
is  done  by  a procedure  called  folding  back  the  decision  dia- 
gram. This  is  a process  of  substituting  values  for  each 
decision  fork  or  act  fork,  beginning  at  the  right-hand  side, 
or  time  horizon,  of  the  diagram.  The  process  of  substitution 
is  continued,  selecting  the  better  of  higher-valued  acts  at 
each  act  fork  subsequent  to  the  initial  decision,  until  the 
decision  diagram  is  simplified  to  the  initial  act  fork.  The 
act  with  the  highest  value  is  the  one  that  should  be  the 
"beat"  decision  in  any  given  situation. 

One  way  of  folding  back  a decision  diagram,  suitable  for 
many  decision  situations,  is  to  fold  back  using  expected 
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value.  The  expected  value  of  an  event  fork  is  the  sum  of  che 
values  produced  by  multiplying  the  value  of  each  possible 
outcome  at  that  event  fork  by  the  probability  of  the  occur- 
rence of  each  outcome.  The  essential  Idea  of  folding  back  by 
expected  value  substitution  can  be  illustrated  by  a simple 
hypothetical  game  (we  shall  return  to  our  bidding  example 
later) . 

suppose  your  decision  is  whether  to  pay  $20  to  get  into  a 
game  where,  if  the  coin  comes  down  heads,  you  win  $100,  and  if 
it  comes  down  tails,  you  lose  $50.  If  you  play,  the  values  of 
the  possible  outcomes  are  +$80  if  you  win,  ($100  win  - $20 
cost  of  entry  * $80)  and  -$70  if  you  lose  (-$50  loss  + -$20 
coat  of  entry  ■«  -$70).  A no-play  decision  has  zero  value  for 
the  outcome.  The  probabilities  of  heads  or  tails  are  of 
course  even,  or  50-50.  Your  decision  options  are  fully  rep- 
resented by  the  decision  diagram  shown  in  Figure  1-llA. 

The  expected  value  of  the  decision  to  play  is  the  value 
of  winning  time's  the  probability  of  winning  ($80  x .5  •»  $40) 
plus  the  value  (or  cost)  of  losing  times  the  probability  of 
losing  (-$70  x .5  - -$35) , or  $5.  Figure  1-llB  shows  the 
equivalent  diagram  after  folding  the  original  diagram  back. 

It  differs  from  Figure  1-llA  only  in  that  a value  of  +$5 
appears  in  place  of  the  win/lose  fork  and  in  the  double  bars 
across  the  no-play  path.  If  you  are  satisfied  that  Figvure 
1-llB  is  indeed  equivalent  to  Figure  1-llA  in  terms  of  the 
relative  attractiveness  of  the  play/no-play  decision,  you  know 
that  solving  the  equivalent  diagram  is  the  same  as  solving  the 
original  diagram.  The  solution  says  you  would  be  Indifferent 
about  winning  $5  for  sure,  and  having  a 50-50  chance  of  win- 
ning $80  or  losing  $70.  Since  winning  $5  for  sure  is  better 
than  winning  nothing,  the  solution  to  the  equivalent  diagram 
is  to  play;  therefore,  the  preferred  decision  in  the  real 
problem  is  also  to  play.  The  double  bars  across  the  no-play 
path  are  a graphic  convention  indicating  a rejected  act  branch. 

Folding  Back  a More  Complex  Tree 

Let's  now  return  to  a more  complex  example,  the  bidding 
problem  which  we  developed  in  earlier  sections  and  which  is 
reproduced  in  Figure  1-12.  To  recapitulate,  the  contractor 
must  decide  whether  or  not  to  bid  on  contract  A,  which  will 
cost  $10,000  if  he  bids.  He  assesses  his  probability  of 
winning  the  contract  as  0.5,  with  a gain  of  $50,000.  He  has 
the  option  at  a later  time  of  bidding  on  contract  B,  at  the 
same  cost.  If  he  has  not  bid  on  contract  A,  the  probability 
of  winning  contract  B is  0.6,  and  the  profit  is  also  $50,000. 

If  he  bids  and  wins  contract  A,  then  he  assesses  the  proba- 
bility of  winning  contract  B to  bu  0.4.  In  this  case,  because 
of  excessive  production  costs  for  both  contracts,  the  profit 
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A.  Original  Diagram 


B.  Equivalent  Diagram 


Figure  1-11 

FOLDED-BACK  DIAGRAM 


on  contract  B will  be  only  $20,000.  Figure  1-12  is  the  com- 
plete decision  diagram  for  solving  the  problem. 

The  first  step  in  folding  back  is  to  eliminate  the  time- 
horizon  event  forks  on  the  right  of  the  diagrain  by  substi- 
tution of  expected  values.  Using  the  same  teohr^-iue  as 
before,  we  can  start  at  W,  the  top  fork  on  the  diagram,  and 
compute  the  expected  value  of  a 40%  probability  of  making 
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NO  BID 
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BID  ON  A 


BIO  ON  B 


BID  ON  B 
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Figure  112 

COMPLETED  DIAGRAM  FOR  BIDDING  DECISION 


$50,000  (+50  on  the  diagram)  and  60%  probability  of  making 
$30,000  (+30).  This  turns  out  to  be  +38.  Now  we  can  elimi- 
nate the  fork  at  W,  and  substitute  the  expected  value  of  +38, 
as  shown  in  Figure  1-13.  The  new  diagram  represents  a situa- 
tion where  the  attractiveness  of  the  immediate  acts  is  exactly 


Figure  1-13 

F»  ST  SUBSTITUTION  ON  BIDDING  DECISION  DIAGRAM 


Proceeding  in  the  same  way,  we  can  substitute  expected 
values  for  each  of  the  other  two  horizon  forks,  as  shown  in 
Figure  1-14A.  It  is  apparent  that  the  decision  problem 
represented  by  Figure  1-14A  should  be  simpler  to  analyze  than 
the  original  problem  of  Figure  1-12.  By  continuing  the 
process  of  substitution,  we  can  make  it  simpler  still. 


PATH 
VALUE 
(in  MOO) 


BID  ON  B 


BID  ON  B 


Figurti  l i/) 

PROGRESSIVE  SUBSTITUTION  OF  DIAGRAMS  IN  BIDDING  DECISION 


In  Figure  1-14A,  the  forks  on  the  right  are  now  all  act 
forks  with  the  indicated  expected  values  as  shown.  The  next 
stage  of  the  substitution  is  very  straightforward  since  we 
should  choose  the  act  with  the  highest  expected  value. 

Taking  the  top  fork  of  Figure  1-14A,  the  choices  are  to  bid 
on  B,  with  an  expected  value  of  +38,  and  not  to  bid,  which  is 
worth  +40.  Since  we  would  pick  +40,  the  other  option  is 
rejected,  as  shown  by  the  double  bars.  Proceeding  similarly 
with  the  other  two  act  forks,  we  can  now  substitute  Figure 
1-14B  for  Figure  1-14A.  This  is  the  result  of  substituting 
for  all  aot  forks  the  value  of  the  better  act. 

The  final  reduction  is  made  by  repeating  the  process  of 

calculating  an  expected  value.  The  event  fork  in  Figure  1-14B 

is  replaced  by  the  expected  value  of  the  50-50  chance  of 
winning  A(+40)  or  losing  A(+15),  which  is  +?.7.5.  This  results 

in  the  diagram  of  Figure  1-14C,  and  we  are  left  with  a single 

act  fork,  with  one  act  valued  at  +27.5  and  the  other  at  +20. 

By  a progression  of  substitutions,  it  is  clear  that  we  should 
bid  on  A,  and  the  folding-back  process  is  completed. 

Folding  baok  by  substitution  of  expected  values  and 
choosing  the  set  with  the  higher  expected  value  is  quite 
routine  and  can  be  easily  computerized.  As  we  have  seen,  the 
best  initial  decision  is  easily  discernible  from  the  simpli- 
fied diagram  resulting  from  the  folding-back  process.  But 
what  of  the  subsequent  acts  or  decisions  in  our  original 
diagram?  Zt  is  often  more  useful,  both  from  a compactness 
standpoint  and  for  a complete  picture  of  the  problem,  to 
retain  the  complete  diagram  and  show  the  values  computed  for 
the  event  forks,  those  chosen  for  the  act  forks  on  the  basis 
of  the  better  act,  and  the  rejected  acts.  Figure  1-15  shows 
the  complete  diagram  with  all  the  substitutions.  The  same 
immediate  act  is  preferred,  bid  on  A. 

Solving  a decision  problem  on  the  basis  of  maximizing 
expected  value  is  often  called  a risk-neutral  strategy.  If 
you  follow  it,  you  will  be  indifferent  between  a decision  to 
do  nothing  and  one  which  gives  you  a 50-50  chance  of  making  or 
losing,  say,  $1,000,000.  A person  controlling  very  large 
resources  may  well  have  such  a risk-neutral  attitude  toward 
decisions  which  represent  relatively  small  gambles  for  him. 

You  or  I,  as  individuals,  would  probably  not  be  risk-neutral 
in  the  face  of  a gamble  which  involved  an  even  chance  of 
winning  or  losing  $1,000,000.  We  would  more  than  likely  be 
averse  to  this  kind  of  a gamble.  This  is  known  as  being 
risk-averse,  and  our  decision,  if  it  is  to  reflect  our  personal 
preferences , would  then  require  a risk-averse  strategy. 

The  insensitivity  of  the  expected-value  approach  to  the 
decision  maker's  attitude  toward  risk  constitutes  a limitation 
on  the  usefulness  of  the  technique.  While  the  use  of  expected 
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FOLDING  BACK  ON  EXPECTED  VALUE 


value  is  quite  appropriate  and  convenient  in  decision  problems 
where  factors  of  risk  do  not  distort  value  considerations,  the 
reader  should  be  wary  of  the  misapplication  of  this  approach. 

In  circumstances  where  biasing  effects  of  risks  do  come 
into  play,  there  are  other  means  of  treating  value  which 
adjust  for  risk  effects.  There  are  techniques  also  for  deter- 
mining quantitative  values  for  qualitative  dimensions  of  worth 
and  for  combining  multiple  dimensions  of  outcome  value  into  a 
single  numerical  expression  of  worth.  These  matters  of  value 
will  be  discussed  in  the  next  chapter. 


iyi<iiiiiii^ijiiy>eiiwi 


CHAPTER  2 


VALUE  AND  EVALUATION 


In  the  preceding  chapter,  we  discussed  how  to  structure 
and  solve  simple  decision  problems.  In  the  examples  used, 
there  was  but  a single  dimension  of  worth  assigned  to  the 
possible  outcomes,  their  monetary  value.  In  practice,  it  is 
rare  to  find  a situation  in  which  outcomes  have  only  a single 
value  dimension.  Usually  there  are  multiple  dimensions  of 
worth  (or  cost)  associated  with  any  outcome.  Furthermore,  as 
will  be  seen,  many  dimensions  of  value  are  qualitative i that 
is,  there  is  no  conventional  metric  to  describe  them.  A 
military  aid  decision,  for  example,  would  be  weighed  not  only 
in  terms  of  quantifiable  economic  considerations  but  in  terms 
of  qualitative  factors  such  as  long-  and  short-term  strategic 
advantage,  the  way  in  which  economic  or  military  aid  would  be 
perceived  by  allied  nations,  and  domestic  political  consider- 
ations. And,  as  was  discussed  at  the  conclusion  of  the  last 
chapter,  there  is  a need  to  handle  values  in  such  a way  that 
risk  may  be  accommodated  in  the  solution  of  a decision  problem. 
Since  decision-analytic  methodology  requires  a single  quan- 
titative expression  of  worth  for  possible  outcomes,  how  then 
does  one  handle  situations  having  multiple  dimensions  of 
value,  many  of  which  are  qualitative  and  all  of  which  can  be 
affected  by  the  risk  inherent  in  the  decision?  There  are  a 
number  of  ways  of  handling  these  complicating  considerations, 
and  all  of  them  in  one  way  or  another  use  a notion  of  equiva- 
lent substitution.  In  this  chapter,  we  shall  present  methods 
for  taking  account  of  risk  in  decision  situations,  for  dealing 
with  multiple  dimensions  of  value,  and  for  assigning  values  to 
qualitative  dimensions  of  worth. 

Folding  Back  by  Substitution  of  Certainty  Equivalents 

Value  for  an  option  (or  any  event  fork)  can  be  thought  of 
as  the  value  (certain)  which  you  would  trade  for  the  xmcertain 
prospect  that  you  face.  In  some  circumstances,  it  is  helpful 
to  think  of  it  as  the  selling  price  of  the  event  fork  treated 
as  a gamble.  The  selling  price  of  the  gamble  may  be  referred 
to  as  a “certainty  equivalent,"  that  is,  an  amount  you  would 
be  just  willing  to  accept  for  certain  in  lieu  of  the  uncer- 
tain prospect  represented  by  the  gamble. 

To  illustrate  how  the  use  of  certainty  equivalents,  which 
reflect  your  attitude  toward  risk, "can  affect  your  decision, 
let's  look  at  the  first  example  we  used  in  Chapter  1 to 
illustrate  the  technique  of  folding  back  a decioiion  diagram. 
Recall  that  you  had  an  opportunity  of  paying  $20  tor  a 50-5C 
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gamble  between  winning  $100  and  losing  $50.  Au  a result, 
either  your  net  gain  would  be  $80,  or  your  net  loss  would  be 
$70.  The  diagram  in  Figure  2-lA  represents  your  decision 
option.  Figure  2-lB  differs  from  Figure  2-lA  only  in  that  a 
value  of  -$7  appears  in  the  place  of  the  win/lose  event  fork. 
What  this  difference  means  is  that  despite  the  expected  value 
of  $5  (and  the  chance  of  winning  $80)  you  would  be  willing  to 
give  up,  say,  $7  rather  than  run  the  risk  of  losing  $70.  If 
the  diagram  is  indeed  equivalent  as  far  as  you  are  concerned, 
it  says  that  you  would  be  indifferent  between  losing  $7  for 
sure,  and  having  a 50-50  chance  of  winning  $80,  or  losing  $70. 

Note  that  this  certainty  equivalent  is  not,  and  in 
general,  does  not  have  to  be  the  same  as  the  expected  value  of 
the  gamble,  which  in  this  case,  is  't$5.  If  you  are  really 
averse  to  risk,  you  might  be  prepared  to  pay  money  to  get  out 
of  a gamble  like  this  one  whose  expected  value  is  positive. 
Since  losing  $7  for  sure  is  worse  than  losing  nothing,  the 
solution  to  the  diagram  in  Figure  2-lB  is  not  to  play; 
accordingly,  the  preferred  decision  in  the  real  problem  is  not 
to  play  either. 

Substitution  of  certainty  equivalents  can  be  used  in  a 
more  complex  decision  diagram  suoh  as  our  bidding  problem 
shown  again  in  Figure  2-2.  Suppose  that  at  W,  you  were  in- 
different between  the  gamble  represented  by  the  event  fork 
and  a value  of  +35  for  sure.  Substituting  +35  for  the  event 
fork  at  W results  in  the  simplified  diagram  shown  in  Figure  2- 
3.  We  have  substituted  for  the  prospect  of  the  original 
gamble,  the  equivalent  value  (certainty  equivalent)  which  is 
equally  attractive,  instead  of  the  expected  value  of  +36, 
which  a risk-neutral  strategy  would  dloiate.  Proceeding  in 
the  same  way,  you  can  substitute  certainty  equivalents  for  the 
other  two  horizon  forks.  You  might  be  left  with  the  diagram 
shown  in  Figure  2-4A.  The  values  substituted  would  Indicate 
that  you  are  slightly  risk-averse  since  the  certainty  equiva- 
lents values  are  somewhat  lower  than  the  expected  values. 

Since,  in  Figure  2-4A,  the  forks  on  the  right  are  now  all 
act  forks  with  certain  values,  you  can  proceed  as  before  by 
substituting  for  all  act  forks  the  value  of  the  better  act. 

The  result  is  shown  in  the  diagram  in  Figure  2-4B. 

To  make  the  final  reduction,  you  simply  repeat  the 
process  you  started  with.  The  act  fork  on  the  right  is 
replaced  by  a certainty  equivalent  which  reflects  the  attrac- 
tiveness of  a 50-50  gamble  between  +40  and  +13.  If  +25  is  a 
value  which  you  find  equivalent,  that  is  the  value  substi- 
tuted. In  the  simplified  diagram  of  Figure  2-4C,  you  are  left 
with  a single  act  fork  with  one  act  valued  at  +25  and  the 
other  valued  at  +18;  and  it  is  clear  which  you  prefer. 
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Figure  2-5  shows  the  complete  decision  diagram,  with  the 
certainty  equivalents  you  have  used.  The  final  valuations  of 
the  immediate  acts  (+25  and  +ld)  are  lower  than  those  obtained 
by  substitution  of  expected  values.  The  same  immediate  act  is 
preferred  (bid  on  A)  and'  the  contingent  decisions  are  not 
changed.  Although  using  certainty  equivalents  confirmed  the 
preference  obtained  by  using  expected  value  in  this  case,  the 
substitution  of  one  for  the  other  does  not  always  yield  the 
same  preference.  A very  risky  gamble  might  be  preferred  to  a 
“sure  thing"  if  you  are  risk-neutral,  but  a “sure  thing"  is 
preferred  if  you  are  risk-averse. 

It  may  seem  that  the  process  whereby  an  equivalent  value 
is  substituted  for  an  event  fork  is  somewhat  arbitrary,  and, 
in  a sense,  it  is.  It  is  a measure  of  how  you  personally  feel 
about  a gamble.  It  may  be  either  higher  or  lower  than  the 
expected  value  that  results  from  calculating  a probability- 
weighted  average  of  the  possible  values  resulting  from  the 
gamble.  If  you  are  risk-averse,  your  certainty  equivalent 
will  be  lower  than  the  expected  value,  as  in  the  case  above. 

On  the  other  hand,  if  you  are  a risk  seeker,  the  certainty 
equivalent  will  be  higher. 

While  substitution  of  certainty  equivalents  has  the  handy 
property  of  taking  into  account  the  decision  maker's  attitude 
toward  risk,  there  is  a major  limitation  to  its  practical  use. 
Requiring,  as  it  does,  node-by-node  assessment  of  certainty 
equivalents,  the  method  can  become  so  laborious  and  time 
consuming  in  large  decision  problems  as  to  be  unmanageable. 
Fortunately,  a methqd  exists  whereby  one  can  achieve  the  same 
advantages  offered  by  substituting  certainty  equivalents  while 
avoiding  the  extensive  labor  of  numerous  asBesaments.  It 
involves  translating  each  of  the  original  values  on  the  diagram 
into  a numerical  expression  of  utility  and  folding  back  the 
decision  diagram  based  on  expected  utility  by  the  same  opera- 
tions previously  presented'  for  expected  value. 

Utility  as  an  Index  of  Worth 

It  should  be  apparent  from  our  discui^sion  of  certainty 
equivalents  that  the  value  of  things  is  a personal,  circum- 
stantial matter  and  that  it  is  these  subjective  values  of 
things  that  are  the  basis  for  decision.  In  the  case  of 
money,  for  example,  it  is  not  necessarily  the  case  that  you 
would  value  a gain  of  $10,000  twice  as  highly  as  a gain  of 
$5,000,  nor  is  a loss  of  $1,000  necessarily  twice  as  severe  as 
a $500  loss.  It  depends  on  the  decision  maker's  personal 
attitudes  and  circumstances,  a dependency  accommodated  by  the 
concept  of  utility. 

Utility  can  be  described  as  a subjective  measure  of 
“liking."  It  is  a personal  value  reflecting  how  you  sub- 
jectively value  something.  For  example,  for  most  families. 
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owning  one  car  would  have  high  utility,  a second  car  somewhat 
less  utility,  and  a third  car  relatively  little  utility.  Thus, 
there  is  a contextual  and  relative  aspect  of  utility.  If  the 
three  cars  are  identical,  there  is  no  difference  between  them 
in  terms  of  objective  value,  but  in  the  context  of  already 
owning  two  cars,  the  utility  of  the  third  car  would  be  sub- 
stantially leas  than  either  of  the  first  two. 

The  contextual  quality  of  the  concept  of  utility  is  an 
interesting  and  valuable  one  for  decision-analytic  applica- 
tions. It  not  only  provides  a means  of  quantifying  subjective 
values,  but  also  provides  a convenient  shorthand  for  accommo- 
dating attitudes  toward  risk  in  the  solution  of  decision 
problems. 

To  illustrate  the  concept  of  utility  and  its  application 
in  a decision  context,  let's  return  to  our  now  familiar 
bidding  example  (Figure  1-10) . Note  that  the  change  in  mone- 
tary assets  at  issue  ranges  from  a loss  of  $20,000  (-$20)  to 
a gain  of  $50,000  (+$50).  In  arriving  at  utilities  in  the 
form  of  a utility  curve  for  a given  value  dimension,  one 
typically  restricts  consideration  to  the  range  of  values 
applicable  to  that  problem.  In  this  case  we  need  to  determine 
utility  values  over  the  range  from  -$20  to  +$50.  These 
values  at  the  upper  and  lower  end  of  the  value  dimensions  are 
called  "reference  values,"  and  are  arbitrarily  given  utilities 
of  0 and  100,  respectively. 

Figure  2-6  presents  a utility  curve  derived  for  the  range 
of  values  at  hand.  The  utility  curve  expresses  the  "degree  of 
liking"  for  each  of  the  range  of  payoffs  shown  along  the 
bottom  of  the  chart.  This  "degree  of  liking"  is  expressed  in 
"utiles,"  shown  on  the  arbitrary  scale  at  the  left  side  of  the 
chart.  To  translate  value  to  utility,  one  moves  vertically 
from  a value  on  the  bottom  of  the  chart  to  the  utility  curve, 
and  from  that  point  of  intersection  directly  across  to  the 
utility  scale.  This  operation  is  shown  by  the  dotted  line 
which  links  a value  of  $0  to  a utility  of  66.  If  you  find  it 
awkward  to  have  a positive  utility  assigned  to  zero  or, 
worse,  to  losses,  you  may  rescale  utility  by  assigning  0 
utility  to  the  66-utile  point  with  increasing  negative  utili- 
ties (0  to  -200  at  -$20,000)  below  it  and  increasing  positive 
utilities  (0  to  100  at  +$50,000)  above  it.  The  scale  can  also 
be  adjusted  by  adding  or  subtracting  a constant  or  by  changing 
the  size  of  the  unit.  The  utility  scale,  as  was  indicated 
before,  is  arbitrary,  as  is  temperature  measured  on  Celcius 
or  Fahrenheit  scales.  A scientist,  for  example,  would  come  to 
the  same  conclusion  if  he  used  a Celcius  rather  than  a Fahren- 
heit scale.  The  choice  of  scale  has  no  effect  upon  the 
relative  utilities,  only  upon  your  convenience  in  assessing 
them. 
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A TYPICAL  RISK-AVERSE  UTILITY  CURVE  FOR  A CHANGE 
IN  ASSETS  FROM  A LOSS  OF  $20,000  TO  A GAIN  OF  $50,000 
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The  utility  curve  shown  in  Figure  2-6  is  arrived  at  by  a 
aeries  of  hypothetical  reference  gambles  presented  bo  the 
decision  maker.  He  night  be  asked,  for  exeuaple,  what  return 
for  certain  he  would  accept  as  being  just  equivalent  to  a 
gamble  offering  him  a 6€%  chance  of  winning  +$50,000  and  a 34% 
chance  of  losing  $20,000.  This  gamble  is  shown  in  Figure  2-7A. 
He  judges  $0  as  his  indifference  point  for  that  gamble;  that 
is,  if  he  were  offered  any  positive  amount  of  money  for  sure, 
he  would  rather  take  that  money  than  risk  the  34%  chance  of 
losing  $20,000  even  though  he  could  win  $50,000.  But  if  he 
had  to  pay  out  any  amount  of  money  for  sure,  he  would  rather 
take  the  gamble.  How  is  that  judgment  used  to  determine  a 
point  on  the  utility  curve?  By  finding  the  utility  of  the 
geunble.  We  do  this  by  calculating  its  expected  utility,  which 
is  a weighted  average  of  utilities  just  as  expected  value  is  a 
weighted  average  of  values.  First,  we  replace  the  values,  +50 
and  -20,  with  their  corresponding  utilities,  100  and  0,  as  in 
Figure  2-7B.  This  geunble  displays  the  decision  maker's  liking 
for  the  outcomes  rather  than  the  actual  outcomes,  100  indica- 
ting most  liking  (for  +50)  and  0 indicating  least  liking 
(for  -20) . Next,  we  reduce  this  gamble  to  a single  equivalent 
utility  by  determining  the  expected  utility  of  the  gamble. 

For  this  gamble,  the  expected  utility  is  (100  x .66)  + (0  x 
.34)  > 66.  Now  we  replace  the  gamble  with  its  expected 
utility,  66,  as  in  Figure  2-7C.  Expected  utilities  are 
utilities,  just  as  an  average  height  of  several  people  is 
itself  a height,  even  if  it  does  not  represent  the  height  of 
nny  one  person.  We  have  now  found  that  the  original  hypo- 
thetical gamble  in  Figtire  2-7A  is  worth  66  utiles  on  a scale 
from  0 to  100.  Since  the  original  gamble  was  judged  to  be 
equivalent  to  $0  for  sure,  it  must  follow  that  $0  and  an 
expected  a utility  of  66  are  equivalent.  We  say  that,  rela- 
tive to  $50  and  -$10,  a value  of  $0  has  a utility  of  66.  The 
utility  of  $0  is  66,  or  $0  is  worth  66  utiles.  Now  we  can 
plot  the  point  which  is  given  by  the  intersection  of  the 
dashed  lines  in  Figure  2-6.  (The  process  of  measuring  utility 
in  this  manner  seems  backwards  to  many  people:  we  chose  a 
gamble  with  a particular  utility  and  then  asked  for  a judgment 
of  an  equivalent  amount  of  money  rather  than  starting  with  a 
particular  amount  of  money  and  finding  its  utility.) 

Next,  we  construct  new  geunbles  with  different  proba- 
bilities and  judge  their  indifference  points.  For  example, 
the  decision  maker  might  be  asked  what  certain  value  he  would 
accept  as  being  equivalent  to  a gamble  which  offered  him  a 90% 
chance  of  winning  $50,000  and  a 10%  chance  of  losing  $20,000. 

In  the  case  of  our  example,  he  declares  a monetary  value  of 
about  $30,000  as  his  indifference  point  for  that  gamble. 
Proceeding  similarly  for  different  g^^nbles,  we  obtain  utility 
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points  corresponding  to  a seunple  of  the  range  of  values  in  the 
bottom  of  the  chart.  With  a sample  of  utility  points  in  hand, 
we  draw  a smoothed  curve  through  them  to  arrive  at  the  deci- 
sion maker's  utility  function  over  the  whole  range  of  values 
relevant  to  the  declaion  problem.  In  effect,  we  have  used  the 
concept  of  certainty  equivalents  developed  In  the  last  section 
along  with  expected  utility  to  derive  the  utility  function. 

It  la  worth  noting  that  If  the  decision  maker  Is  risk- 
neutral,  the  certainty  equivalent  judged  for  any  hypothetical 
gamble  will  equal  the  expected  value  of  the  gamble,  with  the 
result  that  his  utility  function  may  be  represented  by  a 
straight  line  running  diagonally  across  the  chart.  The  fact 
that  the  utility  function  In  Figure  2-6  bends  as  It  does 
reflects  risk  aversion.  For  exaa^le,  the  risk-neutral  value 
(expected  value)  for  the  Implied  gamble  which  our  decision 
maker  valued  at  $0  Ih  Figure  2-7A  Is  (.66  x $50K)  + (.34  x - 
$20K)  ■ $26.2.  Uls  aversion  to  the  risk  Inherent  In  that 
gamble  compelled  him  to  assign  a lower  subjective  value  chan 
the  expected  value. 

A second  useful  and  critical  property  of  utility  curves 
is  that  the  certainty  equivalent  of  any  gamble  within  the 
value  range  can  be  determined  very  straightforwardly  from  Its 
expected  utility  by  reading  across  from  the  expected  utility 
to  the  utility  curve,  and  then  down  to  the  corresponding 
value. 

At  this  point,  we  should  warn  the  reader  that  there  are 
several  acceptable  methods  for  assessing  utility  curves)  the 
procedure  we  have  shown  here,  while  good  for  purposes  of 
explaining  the  meaning  of  utility  curves.  Is  perhaps  not  the 
best  way  to  proceed  In  a practical  problem.  To  determine 
several  points  on  the  curve,  we  presented  several  gambles 
whose  payoffs  were  always  the  same,  tSO  and  -20,  varied  the 
probabilities  and  asked  for  judgments  of  the  certainty  equiva- 
lents. But  because  many  people  find  It  easier  to  think  about 
50-50  gambles,  an  alternative  procedure  Is  to  present  several 
geunbles,  all  with  50-50  probabilities  but  with  different 
payoffs,  and  ask  for  judgments  of  certainty  equivalents. 
Another  method  that  works  well  for  some  problems  Is  to  specify 
the  certainty  equivalent,  the  probabilities,  and  the  payoff  on 
the  lower  branch,  and  ask  for  a judgment  of  the  payoff  on  the 
upper  branch  that  would  be  needed  for  the  decision  maker  to  be 
Indifferent  between  the  certainty  equivalent  and  the  gamble. 

We  would  like  to  make  the  point  that  there  Is  no  one  best  pro- 
cedure for  assessing  utility  curves.  The  method  you  use  in 
any  particular  situation  will  depend  on  the  nature  of  the 
problem  and  the  experience  and  background  of  the  decision 
maker. 
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Once  a utility  curve  has  been  defined  for  the  range  of 
values  relevant  to  the  decision  problem  under  consideration, 
we  are  in  a position  to  facilitate  greatly  the  solution  of  the 
problem.  The  reason  is  that  by  using  utilities  as  the  outcome 
values,  we  can  fold  back  the  decision  diagram  using  expected 
utility  (calculated  the  same  way  as  expected  value) . Expected 
utility  is  equal  to  the  sum  of  products  o^  the  probabilities 
and  utilities  on  the  event  branches  of  a particular  fork. 

Figure  2-8  shows  a diagram  of  our  bidding  example  where 
utilities  from  the  curve  in  Figure  2-6  have  been  substituted 
for  the  original  dollar  values.  Folding  back  on  utilities,  as 
in  Figure  2-9,  shows  that  bid  on  A has  the  higher  expected 
utility  (79)  and  should  be  the  preferred  act.  Again,  although 
this  act  is  also  the  preferred  act  in  the  risk-neutral  analy- 
sis previously  conducted  with  expected  value,  it  is  important 
to  realize  that  the  two  solutions  need  not  have  been  the  same. 
The  difference  in  utility  between  bidding  on  A and  not  bidding 
is  small,  only  2 utiles  (79-77).  A slightly  more  risk-averse 
utility  curve  (more  bowed  upward)  would  result  in  a higher 
expected  utility  for  not  bidding  and  thus  reverse  the  result 
of  the  expected  value  analysis. 

Utilities  Applied  To  Non-Monetarv  Values 

The  device  of  handling  risk  aversion  through  utilities  is 
not  restricted  to  problems  where  the  values  are  monetary  (or 
where  non-monetary  considerations  have  been  translated  into 
monetary  units) . Suppose,  for  example,  that  you  are  eval- 
uating a Mideast  foreign  policy  Inltia  ;!ve  > hose  primary 
purpose  is  to  relieve  potential  u.s.  energy  shortages  in  the 
medium  term. 

Analysis  has  proceeded  as  follows:  the  main  criterion 
chosen  is  an  "energy  shortage  index,"  measured  according  to  a 
complex  formula  which  takes  into  account  the  pattern  of 
supply  and  demand  over  time  for  oil  (or  its  equivalents) , and 
which  is  adjusted  for  any  political  side  effects.  Considera- 
tion of  possible  contingencies  has  been  displayed  in  summary 
form  so  that  a stripped-down  decision  diagram  might  look  like 
that  in  Figure  2-10. 

If  the  decision  maker  is  risk-neutral  in  terms  of  the 
energy  shortage  index,  then  routine  folding  back  based  on 
expected  value  (with  probabilities  given  in  the  diagram)  shows 
the  "treaty"  decision  to  have  a lower  (i.e.,  better)  expected 
shortage  than  "no  treaty"  (30  vs.  40) . It  would,  on  that 
basis,  be  preferred.  On  the  other  hand,  if  the  result  of 
a bad  shortage  is  very  serious,  the  decision  maker  would 
probably  be  risk-averse.  He  might  then  proceed  by  assessing  a 
utility  curve  like  the  one  shown  in  Figure  2-11.  (Note  that 
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It  slopes  down  from  left  to  right  because  more  shortage  means 
less  utility.)  In  this  case,  since  the  expected  utility  of 
"treaty"  proves  to  be  the  less  attractive  (58  vs.  68),  the  "no 
treaty"  option  is  now  preferred.  (Under  risk  neutrality  as 
calculated  by  expected  value,  the  decision  went  the  other 
way. ) 


It  can  be  denvonstrated  that  in  most  decision  situations, 
solving  the  decision  problem  in  terms  of  expected  utility 
instead  of  expected  value  leads  to  the  best  solution  from  the 
decision  maker's  perspective.  , 

Why  You  Should  Maximize  Expected  Utility 

The  following  discussion  is  concerned  with  persuading  you 
that  the  preceding  procedure  for  using  utilities  (defined  in 
terms  of  reference  gambles)  is  logically  sound.  We  shall  use 
a concrete  example  to  demonstrate  that  we  act  with  the 
highest  expected  utility  is  the  one  that  should  be  chosen. 
Assume  that  you  have  been  offered  the  choice  of  the  two 
gambles  illustrated  in  Figure  2-12. 

For  Gamble  A,  a fair  coin  will  be  tossedii  if  it  comes  up 
heads,  you  will  win  $7,000  oashi  and  if  it  comes  up  tails,  you 
will  win  nothing.  For  Gamble  B,  a fair  coin  will  also  be 
tossed)  if  it  comes  down  heads,  you  will  win  $4,000;  and  if  it 
comes  down  tails,  you  will  win  $1,000.  The  problem  is  to 
decide  which  of  these  two  gambles  you  should  take.  You  are 
offered  either  one  of  them,  but  only  one.  Gamble  A is  worth 
$1,000  more  in  terms  of  expected  value  ($3,500  vs  $2,500),  but 
that  does  not  necessarily  mean  that  you  prefer  Gamble  A. 

Let  us  further  assume  that  you  are  quite  xisk-averr.e, 
that  you  do  not  like  to  take  serious  chances  with  money.  In 
fact,  assume  that  the  degree  to  which  you  are  risk-averse  is 
described  by  the  utility  function  in  Figure  2-13.  The  refer- 
ence values  are  $0  and  $7,000,  and  amounts  in  between  repre- 
sent increases  on  your  current  scale. 

We  can  now  use  this  utility  function  and  translate  the 
values  on  the  decision  diagram  as  shown  in  Figure  2-14.  For 
this  diagram  the  end  points  are  measured  in  utiles  rather  than 
dollarsi  the  utility  of  $7,000  is  100;  of  $0,  0;  of  $4000, 

84}  and  of  $1000,  50.  The  expected  utilities  are  SO  if  you 
choose  Gamble  A,  and  67  for  choosing  Gamble  B. 

Until  now,  we  have  been  following  the  practice  of  solving 
a decision  diagram  by  folding  it  back.  Now,  however,  we  are 
going  to  move  in  the  opposite  direction.  We  are  going  to 
expand  the  diagram  rather  than  simplify  it.  We  shall  sub- 
stitute a gamble,  an  event  fork,  for  each  of  the  end  points  of 
the  diagram  in  Figure  2-14  in  order  to  create  the  new  decision 
diagram  shown  in  Figure  2-15. 
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UTILITY  ANALYSIS  OF  COIN-TOSSING  EXAMPLE 


There  are  four  end  points  In  Figure  2~14  and  each  has 
been  converted  into  a corresponding  event  fork  in  Figure  2- 
15.  The  event  fork  is  constructed  so  that  $7r000f  the 
maximum  ^ain,  is  associated  with  the  probability  (in  percent) 
equal  to  the  utility  of  the  end  point.  Otherwise/  $0  is 
won.  Accordingly/  the  utility  of  100  on  the  top  branch  in 
Figure  2-14  is  equivalent  to  the  top  event  fork  in  Figure  2- 
15/  where  $7,000  is  won  with  probability  1.0  and  $0  is  won 
with  probabii*  ty  0.  Progressing  down  to  the  next  end  point/ 
with  utility  .0/  we  substitute  the  event  fork  in  which  $7,000 
is  won  with  probability  0 and  $0  is  won  with  probability 
1.0.  The  probabilities  of  the  extended  branches  following 
Act  A are  always  1 or  0 because  the  utilities  of  the  end 
point  are  100  and  0,  respectively. 

Now  consider  Gamble  B.  Here  a utility  of  84  is  equiva- 
lent to  an  event  fork  in  which  $7,000  is  won  with  proba- 
bility 0.84  and  $0  with  probability  0.16.  Finally,  for  the 
bottom  end  point  with  the  utility  of  50,  we  have  an  equiva- 
lent gamble  in  which  $7,000  is  won  with  probability  0.50  and 
$0  with  probability  0.50. 

The  nxjDbera  in  the  parentheses  to  the  right  of  Figure 
2-15  indicate  the  path  probabilities.  For  the  top  branch, 
the  path  probability  is  .50  x 1.0,  or  is  .50.  In  each  case, 
the  path  probability  is  the  product  of  the  event  probabilities 
which  comprise  the  path. 

We  now  have  a rather  complicated  decision  diagram,  but 
one  with  only  two  possible  outcomes t you  either  win  $7,000 
or  you  win  $0.  For  each  act,  we  now  combine  all  branches 
with  identical  consequences  and  add  the  path  probabilities 
for  the  identical  payoffs.  The  result  is  a simpler  but 
equivalent  decision  diagram  with  two  possible  consequences, 
win  $7,000  or  win  $0,  following  each  act.  This  simpler 
decision  diagram  is  displayed  in  Figure  2-16.  For  Gamble  A, 
you  have  simply  ended  up  with  the  original  gamble.  You  win 
$7,000  with  a probability  0.5,  or  you  lose  $0.  But  for  Act  B, 
you  have  a very  different  gamble  from  the  original  one. 

Here,  you  win  $7,000  with  probabi  ty  0.67,  or  you  lose  $0 
with  probability  .33.  Note  that  . .e  probability  of  winning 
$7,000  is  expressed  as  a percentage  for  each  gamble,  then 
the  percentage  is  equal  to  the  expected  utility  of  that 
g£unble.  Since  you  presumably  prefer  a 67  percent  chance  of 
winning  $7,000  to  a 50  percent  chance  of  winning  the  same 
amount.  Gamble  B is  preferred  to  Gambls  A.  Our  previous 
analysis  of  Figure  2-14  showed  that  50  is  the  expected 
utility  of  Gamble  A,  and  67  is  the  expected  utility  of 
Gamble  B.  The  preferred  gamble  is  the  one  with  the  higher 
expected  utility. 
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This  is  the  key  point  of  the  demonstration.  It  is  always 
possible  to  substitute  a new  diagram  that  is  equivalent  to  the 
original  decision  diagram  in  the  manner  described  above.  This 
new  diagram  will  have  the  property  that,  for  any  initial  act, 
you  win  the  maximum  aiiiount  with  a probability  (expressed  as  a 
percentage)  equal  to  the  expected  utility  of  the  initial  act 
and  you  win  the  minimum  amount  otherwise.  In  this  substitute 
diagram,  we  no  longer  find  it  necessary  to  be  concerned  with 
trade-off o between  probabilities  and  amounts  to  be  won, 
because  the  amounts  to  be  won  are  held  constant  across  all 
initial  acts.  You  either  win  the  maximum  amount  or  you  win 
the  minimum  amount.  What  varies  from  act  to  act  is  only  the 
probability  of  winning,  and  that  probability  (expressed  as  a 
percentage)  la  equal  to  the  expected  utility  associated  with 
the  act.  It  must  be  true  that  any  decision  maker  faced  with 
two  probabilities  of  winning  the  maximum  amount  will  select 
the  gamble  with  the  higher  probability  of  winning.  It  follows 
that  he  should  also  select  the  initial  act  with  the  higher 
expected  utility.  In  a somewhat  anecdotal  fashion,  we  have 
now  demonstrated  what  we  set  out  to  show. 

Multiple  Attributes  of  Value 

So  far,  we  have  considered  decision  situations  wherein 
there  has  been  but  one  attribute  of  value.  In  practice,  it  is 
rare  to  find  decision  situations  where  only  one  value  attri- 
bute bears  on  a decision.  A decision  can  often  be  dominated 
by  one  attribute  but  it  is  rare  indeed  to  encounter  a complex 
decision  problem,  one  which  would  merit  the  effort  of  serious 
analytical  effort,  that  has  but  one  relevant  value  attribute. 
It  is  also  the  case  that  many  value  attributes  which  must  be 
considered  in  decisions  are  qualitative,  that  is,  are  not 
expressed  in  any  conventional  metric.  In  the  following 
sections,  we  shall  present  various  methods  to  handle  multiple 
attributes  of  value  in  a decision-analytic  context  and  present 
additional  means  to  handle  qualitative  value  attributes. 

Measuring  Different  Attributes  by  a Single  Criterion 

One  approach  is  to  select  one  readily  quantified  cri- 
terion, like  cost  in  dollars,  and  express  all  other  attributes 
in  terms  of  that  one.  This  procedure  can  best  be  explained  in 
the  context  of  an  example.  Suppose  you  are  invited  to  play  a 
coin  tossing  game  which  leaves  you  either  $80  ahead  or  $70 
behind  with  equal  probability.  Now  further  suppose  that 
winning  a game  like  this  would  give  you  some  measure  of 
gratification  beyond  the  monetary  reward  but  that  getting  into 
a game  like  this  and  loning  carries  a special  cost  to  you,  for 

example,  embarrassmen't  in explaining  the  loss  to  your  spouse. 

Your  real  perception  of  the  decision  can  be  represented  by  the 
diagram  in  Figure  2-17A. 
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Suppose  you  would  not  exchange  the  gratification  of 
winning  for  less  than  $5,  but  you  would  pay  $20  to  avoid  the 
embarrassment  of  losing.  These  judgments  allow  you  to  sub- 
stitute problem  shown  in  Figure  2-17B  which  has  only  one 
quantiiM^'^a  criterion,  and  which  therefore  can  be  analyzed  by 
foldinc  exactly  as  before. 

In  principle  it  is  always  possible  to  measure  different 
attributes  by  a single  quantified  criterion  whether  or  not  all 
of  them  are  numerically  measurable.  What  is  important  is  that 
the  resulting  values  correctly  reflect  differences  among 
outcomes  in  terms  of  their  relative  importance  in  the  decision 
making  process.  This  principal  is  further  illustrated  in  the 
following  example. 

A presidential  aide  is  faced  with  the  decision  of  whether 
or  not  to  recommend  signing  a political  and  economic  treaty 
with  a Mideast  country.  The  main  motivation  is  to  ensure  a 
higher  level  of  oil  supplies  over  the  next  ten  years.  How- 
ever, it  is  also  important  to  the  U.S.  to  take  account  of  the 
price  to  be  paid  for  the  oil  and  the  impact  of  the  treaty  on 
political  goodwill  vis-a-vis  several  groups  of  countries. 

Since  the  possible  outcomes  for  either  decision  are  very 
complex,  the  aide  selects  a limited  number  of  possible  sce- 
narios to  represent  the  real  possibilities,  assigns  proba- 
bilities to  them,  and  notes  what  their  impact  on  each  of  the 
several  attributes  would  be.  Figure  2-18  shows  a simplified 
version  of  the  decision  diagram.  Note  that  the  aide  assigned 
a slightly  higher  probability  to  scenario  A,  leading  to  5 
million  barrels  a day,  for  the  treaty  than  for  no  treaty. 

He  then  evaluates  the  "trade-off"  between  the  main  attri- 
butes of  oil  supply  and  each  other  attribute  in  terms  of  oil 
supply,  as  shown  in  Figure  2-19.  These  measurements  take  into 
account  not  only  his  evaluation  of  the  qualitative  attributes, 
but  also  the  importance  of  oil  in  the  light  of  developments  in 
the  pattern  of  supply  and  demand  of  energy  for  the  U.S.  indi- 
cated in  the  scenario. 

The  aide  nets  out  the  values  and  folds  back  on  equivalent 
value.  The  result,  shown  in  Figure  2-19,  shows  that  the  aide 
should  recommend  rejecting  the  treaty  for  the  particular 
evaluations  and  assessments  he  has  made.  It  further  shows 
that  he  would  need  to  be  guaranteed  a bonus  of  1.6  million 
barrels  of  oil  a day  (the  difference  between  -0.5  bbl  per 
calendar  day  and  +1.1  bbl  per  calendar  day)  before  he  would 
reach  an  indifference  point  between  having  a treaty  or  not. 

A Weighted  Index  of  Attractiveness 

Where  no  quantitative  criterion  dominates  or  the  quali- 
tative attributes  are  so  critical  as  to  swamp  differences  in 
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DECISION  DIAGRAM  FOR  OIL  TREATY  WITH  MULTIPLE  CRITERIA 


the  main  criterion,  it  may  be  difficult  to  make  the  required 
measurements  with  an  adequate  degree  of  precision.  In  such 
cases,  as  an  alternative  to  the  single  criterion  approach  to 
multi-attributed  decisions,  an  arbitrary  weighted  index  of 
attractiveness  can  be  used. 

Suppose  that  the  decision  Is  whether  or  not  to  close 
down  a military  installation.  In  considering  the  possible 
outcome  of  his  decision,  the  decision  maker  might  take  into 
account  the  following  attributes:  contribution  of  the 
installation  to  stirategic  objectives,  goodwill  in  the  local 
population,  and,  to  jB  lesser  degree,  cost. 
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DECISION  DIAGRAM  FOR  OIL  TREATY  WITH  SINGLE,  QUANTIFIED  CRITERION 
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If  uncertainty  is  not  an  issue,  the  choice  might  be 
represented  as  in  Figure  2-20.  This  representation  reflects 
the  judgment  that  while  closing  the  Installation  will  repre- 
sent an  important  sacrifice  in  terms  of  national  strategic 
objectives,  it  will  have  a favorable  impact  on  relations 
with  the  local  population  and  will  save  $50  million  a year. 
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To  construct  a weighted  index,  we  first  assign  an  arbi- 
trary scale  for  each  attribute,  say,  from  0 to  100,  defining 
carefully  the  ends  of  the  scale.  For  example,  0 value  of 
the  strategic  attribute  might  be  a state  of  national  emer- 
gency, and  100  might  be  the  practical  ultimate  in  preparedness. 
Next,  each  decision  option  is  valued  on  each  attribute 
scale,  and  a measure  of  relative  Importance  between  attri- 
butes is  set  up,  as  shown  in  Figure  2-21A. 

Finally,  the  attribute  values  are  weighted  and  summed, 
as  shown  in  Figure  2-21B,  to  give  a single  index  of  attrac- 
tiveness, which  indicates  that,  on  this  set  of  value  assess- 
ments, not  closing  the  installation  is  preferred. 

I In  this  simplified  hypothetical  example,  the  formal 

diagram  shown  in  Figure  2-21  may  appear  to  add  little  to  the 
initial  statement  of  the  problem  in  Figure  2-19  which,  no 
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DECISION  DIAGRAM  FOR  CLOSING  DOWN 
A MILITARY  BASE  USING  VVEIGHTED-SUM  PROCEDURE 
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doubt#  could  be  readily  digested  intuitively.  However,  when 
the  number  of  options#  possible  outcomes#  and  attributes  is 
multiplied#  the  case  for  this  variant  of  "divide  and  conquer" 
becomes  nuire  compelling. 

It  should  be  noted  that  this  weighted-^sum  procedure#  in 
contrast  to  the  previous  adjustment  procedure#  is  not  always 
logically  sound#  if#  for  example#  there  are  important  inter- 
actions among  criteria.  There  are  other  procedures  which  are 
logically  watertight#  but  they  are  typically  less  convenient 
to  apply. 

Multi-Attribute  Utility  (MAU)  Models  for  Evaluating  Complex 
Systems 

The  fact  that  utility  values  provide  an  index  of  merit 
for  things#  or  multiples  of  things#  and  also  provide  a common 
metric  for  aggregating  the  measures  into  a single  index  of 
worth  makes  the  concept  of  utility  highly  useful  in  evaluating 
complex  systems.  Determining  a figure  of  merit  for  alter- 
native complex  system  designs  can  be  a formidable  task  largely 
because  the  competing  system  designs  vary  from  each  other  in 
terms  of  many  system  attributes  bearing  on  overall  worth. 
Proposed  radio  set  A#  for  example#  may  be  lighter  than  B but 
have  less  transmission  range.  It  may  be  more  reliable  but 
have  fewer  channels.  It  may  require  less  power  but  be  more 
readily  jammed  and  so  on.  To  further  compound  the  problem# 
the  system  under  consideration  may  face  multiple  roles  (e.g. 
battle  communication#  rear-area  use)  and  operating  environ- 
ments (urban  area#  open  desert) . Which  design  for  which  role 
for  which  environment?  Which  would  be  the  best  design  for 
all? 


A variety  of  procedures  can  bo  used  to  make  the  indicated 
evaluation.  These  could  range  from  overall  (global)  judgments 
by  experts  as  to  which  system  is  best  to  the  use  of  vevy 
complex  simulations.  The  former  approach  tends  to  be  too 
simplistic  and  subject  to  many  inherent  difficulties,  one  of 
which  is  that  the  evaluation  problem  is  usually  far  too  com- 
plex for  accurate#  global  judgments;  the  latter#  though  often 
used#  suffers  from  the  shortcoming  that  resulting  performance 
measures  may  not  be  directly  related  to  measures  of  worth  or 
utility  for  mission  performance.  By  using  utility  as  a 
measure  of  merit  and  by  applying  this  concept  to  a systematic 
disaggregation  of  the  total  problem,  we  can  circumvent  the 
shortfalls  of  global  judgments  and  simulation  approaches. 

This  approach  is  known  as  multi-attribute  utility  (MAU) 
assessment.  How  it  works  in  practice  will  be  shown  in  the 
following  example#  which  is  a partial  representation  of  an 
evaluation  of  alternative  military  radio  systems  that  was 
actually  conducted#  but  which  is  here  greatly  simplified  for 
purposes  of  exposition. 
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The  evaluation  problem  involves  determining  the  military 
worth  of  four  alternative  field  radio  designs  so  that  a 
rational  selection  could  be  made  among  them.  In  the  MAU 
approach,  the  first  step  Is  to  establish  a model  structured  to 
relate  physical  system  parameters  to  military  value.  This 
structure  is  a representation  of  how  the  physical  and  per- 
formance attributes  of  the  system  or  systems  being  evaluated 
are  translated  into  measures  of  utility,  which  in  turn  reflect 
how  well  each  system  accomplishes  its  military  mission.  In 
the  case  of  the  radio  evaluation,  the  system  attributes  are 
first  disaggregated  into  two  major  categories:  technical 
system  utility  (how  well  each  system  functions  as  a radio)  and 
operational  acceptability  (how  attractive  the  system  is  from 
an  operational  point  of  view).^  Each  of  these  two  major 
assessment  categories  is  further  disaggregated,  as  shown  in 
Figures  2-22A  and  2-22B.  The  disaggregation  proceeds  until, 
at  the  lower  level  of  the  model,  the  system  is  depicted  in 
terms  of  its  actual  measurable  attributes.  These  measurable 
attributes  serve  as  the  referent  for  the  assessment  of  utility 
functions  for  those  system  attributes.  These  utility  curves 
are  elicited  in  the  form  of  judgments  by  subject  area  experts 
and  users. 

After  the  utility  functions  have  been  developed,  it  is 
necessary  to  assign  importance  weights  to  each  branch  of  the 
model.  Two  kinds  of  importance  weights  are  required,  de- 
pending upon  the  rules  of  combination.  In  the  case  of  a set 
of  attributes  whose  relationship  is  captured  by  an  additive 
rule,  importance  weights  simply  provide  a means  of  calculating 
a weighted  average  over  the  attributes  being  considered. 
Consequently,  importance  weights  assume  the  form  of  coeffi- 
cients that  are  multiplied  by  the  utility  of  each  of  the 
attributes,  and  these  coefficients  sum  to  1.0  across  all 
related  attributes  within  a given  level  of  the  hierarchical 
structure.  The  weight  given  to  a particular  attribute  re- 
flects the  relative  contribution  of  that  factor  vis-a-vis  the 
contribution  of  the  other  related  attributes  to  utility  for 
that  level  of  the  model. 

There  are  two  sets  of  multiplicative  factors  in  Figure 
2-22A.  If  factors  within  a given  level  are  multiplicative 
(that  is,  they  are  Interactive  in  a value-wise  sense) , then 
the  measure  of  utility  of  any  factor  may  be  considered  as  a 
measure  of  degradation.  For  example,  technical  performance 
and  dependability  are  shown  as  interacting  in  Figure  2-22A. 
Systems  capable  of  great  technical  performance  are  more  com- 
plex than  limited-capability  systems,  and  so  are  less  depend- 
able since  more  things  can  go  wrong.  When  variables  interact 
in  this  manner.  Importance  weights  have  the  effect  of  re- 
scaling the  attributes.  Prior  to  applying  a weight,  the 

The  distinction  is  real.  For  example,  a radio  system  may 
function  well  as  a radio  but  be  too  heavy  to  carry  in  the 
field. 
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utility  over  one  of  the  attributes  may  have  any  value  between 
zero  and  one.  However,  assume,  for  example,  when  this  attri- 
bute takes  on  its  worst  value,  that  the  effectiveness  of  a 
system  is  degraded  to  the  order  of  30%.  In  this  case,  the 
utility,  originally  scaled  from  0 to  1.0,  would  be  rescaled 
from  0.3  to  1.0. 

Having  now  achieved  weighted  utilities  for  the  elements 
of  the  model,  the  final  step  is  to  combine  the  values  in 
accordance  with  the  appropriate  combination  rule  (in  our  case, 
a combination  of  additive  and  multiplicative  rules)  to  arrive 
at  utilities  for  successively  higher  levels  of  aggregation. 

The  final  utility  value  of  a radio  assessed  in  our  sample 
model  would  be  arrived  at  by  adding  the  weighted  utility  value 
of  technical  system  utility  (Figure  2-2 2A)  to  that  of  opera- 
tional acceptability  (Figure  2-22B) . Utility  value  thus 
derived  for  each  of  the  system  designs  under  consideration 
provides  a systematically  derived,  numerical  basis  for  com- 
parison. 

The  multi-attribute  utility  approach  described  above  has 
a number  of  advantages.  First,  it  permits  an  individual  who 
is  an  expert  in  a particular  area  to  make  judgments  which 
involve  his  particular  area  of  expertise  rather  than  making  an 
overall  judgment  of  worth  which  may  fall  outside  his  area  of 
expertise.  Second,  disaggregating  the  judgments  of  individual 
experts  provides  an  explicit  trail  leading  from  measures  of 
system  performance  to  measures  of  benefit  or  utility.  Thus, 
the  judgments  are  public  rather  than  private  and  are  subject 
to  screening. 

Nonetheless,  there  are  many  critics  who  bridle  at  the  use 
of  subjective  values  no  matter  how  systematically  or  logically 
they  might  be  used.  They  contend  that  the  subjective  impor- 
tance weights  might  be  wrong,  the  assessed  utilities  inaccurate, 
or  the  logic  of  the  model  structure  flawed.  While  it  is  true 
that  such  errors  can  occur,  there  are  methods  for  determining 
what  Influence  on  final  assessments  such  errors  might  have. 

These  methods,  called  sensitivity  analyses,  can  be  used  to 
check  the  sensitivity  of  the  model  to  variances  in  subjective 
values,  to  compare  the  assessments  made  by  the  model  to  firm 
external  reference  criteria,  and  to  permit  further  considera- 
tion of  assessments  crucial  to  the  output  of  the  model. 

Sensitivity  analyses  within  the  model  involve  an  evalua- 
tion of  the  degree  to  which  changes  in  utility  functions, 
importance  weights,  and  combination  rules  influence  the 
output  of  the  model.  For  example,  to  what  degree  does  the 
relative  utility  of  Alternative  System  1 vs  Alternative 
System  2 depend  upon  the  weights  assigned  to  an  attribute 
within  the  model  or  to  different  utility  values?  Sensitivity 
analyses  external  to  the  model  provide  a means  of  calibrating 
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the  model  and  involve  the  use  of  the  model  to  compare  known, 
well-understood  alternative  systems  before  using  the  model  to 
evaluate  less  well-understood  systems.  This  is  to  say  that  it 
is  useful  to  test  the  model  against  educated  judgments  about 
the  relative  utility  of  existing  systems/  or  components  of 
existing  systems/  before  applying  the  model  to  possible  future 
systems.  If  the  model  appears  to  be  blatantly  at  odds  with 
expert  judgments  pertaining  to  the  relative  utility  of  known 
systems/  analysis  of  the  model  should  lead  to  Identifying  the 
source  of  the  disagreement  and  thereby  facilitate  modification 
of  the  model  In  order  to  bring  subsequent  evaluations  more 
into  line  with  informed  judgments.  It  is  difficult  to  over- 
emphasize the  Importance  of  these  kinds  of  sensitivity  analyses 
In  terms  of  both  Improving  the  evaluation  model  Itself  and 
developing  confidence  and  understanding  on  the  part  of  the 
user  who  Intends  to  make  recommendations  based  upon  the  output 
of  the  model. 
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CHAPTER  3 


MEASURING  UNCERTAINTY 


In  this  chapter  we  provide  some  useful  background  infor- 
mation about  probabilities,  describe  ways  of  obtaining  proba- 
bility values  as  a numerical  expression  of  uncertainty, 
discuss  the  biases  that  may  result  during  the  assessment  of 
probabilities  (imd  how  to  compensate  for  them),  and  finally, 
consider  how  to  revise  probability  values  on  the  basis  of 
new  infcnuaticn. 


Qualitative  Expressions  of  Uncertainty 


Managers  are  accustomed  to  dealing  with  uncertainty 
informally.  For  instance,  they  may  prepare  contingency  plans 
to  be  put  in  effect  if  some  uncertain  future  event  turns  out 
differently  from  what  is  expected.  The  term  "calculated  risk" 
is  sometimes  used  loosely  in  this  context  to  indicate  that 
uncertainties  have  to  be  considered.  In  the  decision-analytic 
approach  to  problem  solving,  however,  the  amount  of  risk 
(equal  to  the  sum  of  the  outcomes  times  their  probabilities) 
actually  is  calculated.  Many  managers  also  express  their 
feelings  aEbut  future  events  by  using  semiquantltative  terms 
such  as  "unlikely,"  "probable,"  or  "rare."  For  example,  a 
marketing  person  might  discuss  next  year's  projected  sales  for 
a product  by  citing  t’r.ree  different  possible  levels  of  volume: 
optimistic,  pessimistic,  and  best  guess.  Similarly,  a mili- 
tary analyst  might  respond  to  a question  about  whether  an 
attack  will  occur  at  a particular  time  and  place  by  saying  it 
is  "highly  unlikely."  Figure  3.1  shows  these  two  examples  in 
decision  tree  format.  Before  these  nodes  can  become  part  of  a 
decision  analysis,  however,  both  the  uncertainties  and  the 
values  of  the  outcomes  must  be  expressed  numerically.  Chapter 
2 explained  how  the  relative  desirability  of  different  out- 
comes can  be  translated  into  the  measure  called  utility.  How 
we  will  discuss  how  to  quantify  uncertainty  in  terms  of  a 
probability.  This  process,  called  "probability  assessment," 
is  a collection  of  techniques  that  help  a person  take  feelings 
about  uncertainty  that  may  be  vaguely  but  comfortably  expressed 
in  qualitative  terms  and  convert  these  findings  into  numbers 
on  a scale  from  zero  to  one.  An  outcome  which  is  impossible 
will  have  a probability  equal  to  zero.  For  example,  the 
probability  that  the  Soviets  have  a missile  which  exceeds  the 
speed  of  light  is  zero.  On  the  other  hand,  if  an  outcome 
is  absolutely  certain  to  occur,  it  will  have  a probability 
equal  to  one.  For  example,  the  probability  that  the  tem- 
perature in  downtown  Nairobi  tomorrow  will  be  higher 
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Sthan  absolute  zero  is  1.0.  Intermediate  values  of  proba- 
!,  bility  represent  intermediate  degrees  of  certainty.  A 

! probability  of  0.5  means  that  the  event  is  just  as  likely  to 
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EXAMPLES  OF  EVENT  NODES  WHERE 
UNCERTAINTY  IS  EXPRESSED  IN  SEMI-QUANTITATIVE  TERMS 


Definition  of  Probability 
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The  probability  values  we  shall  elicit  represent  a per' 
son's  degree  of  belief  that  various  events  will  occur.  The 
source  of  this  information  may  be  the  manager,  an  expert 
called  in  by  the  manager,  or  even  a group  of  people.  The 
idea  of  a probability  as  a degree  of  belief  contrasts  with 
some  more  narrow  definitions  of  probability,  for  instance, 
probability  as  a relative  frequency  or  probability  derived 
from  arguments  of  symmetry.  Examples  of  the  latter  two 
types  of  probabilities  would  be: 

1.  Relative  Frequency.  A flight  attendant  asks 
passengers  whether  they  want  pan-fried  chicken  or  Salisbury 
steak  for  dinner.  By  the  time  50  passengers  have  been 
asked,  the  responses  are  chicken,  30  passengers;  steak,  20 
passengers.  Therefore,  he  estimates  that  the  probability 
that  the  next  person  will  say  "chicken"  is  30/50  or  0.6. 
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2.  Syinmetry  Argument.  A carefully  made  "Las  Vegas" 
die  (perfectly  syminetricai  and  made  of  homogeneous  material) 
is  rolled.  Because  of  its  physical  characteristics,  each  of 
the  six  sides  is  equally  likely  to  be  on  top  at  the  end  of 
,,  the  roll.  The  probability  that  a particular  number,  say, 

I five,  will  show  is  equal  to  1/6  or  0.1667. 

Neither  of  these  narrow  methods  for  defining  probability 
is  very  useful  for  the  manager  because  a manager  is  rarely 
concerned  with  a situation  where  exactly  the  seune  event 
occurs  over  and  over  so  that  relative  frequency  can  be  used, 
or  with  a situation  where  a number  of  events  being  considered 
can  be  shown  to  be  equally  likely  to  occur.  Consequently,  we 
accept  the  broader  definition  of  probability  as  a degree  of 
belief  and  concentrate  on  ways  of  finding  out  the  appropriate 
numerical  values  to  express  this  degree  of  belief.  These 
values  express  what  are  called  personal  probabilities . 


Probability  Rules 


Probability  theory  has  as  its  foundation  a number  of 
mathematical  rules  about  the  definition  of  probabilities  and 
computations  Involving  probabilities.  Xn  this  introduction  to 
decision  analysis,  however,  only  a few  of  these  features  of 
probability  need  to  be  discussed. 

Range  of  Probability  Values 

We  have  already  mentioned  that  probabilities  must  lie 
somewhere  in  the  range  from  O.po  through  1.00.  According  to 
this  convention,  it  would  be  meaningless  to  say  that  some 
outcome  had  a negative  probability  or  a probability  greater 
than  1.0. 

Mutually  Exclusive  Outcomes 

The  set  of  outcomes  of  an  event  should  be  chosen  or 
defined  so  that  different  outcomes  are  mutually  exclusive, 
that  is,  so  that  no  more  than  one  of  the  outcomes  in  the  set 
can  occur.  In  the  military  prospects  exiunple,  two  outcomes 
are  considered:  enemy  attack  and  no  attack.  Since  only  one 
of  these  can  occur,  the  events  are  mutually  exclusive. 

Exhaustive  Outcomes 

The  outcomes  of  an  event  should  be  chosen  or  defined  so 
that  they  are  exhaustive  in  the  sense  that  some  one  of  the 
outcomes  must  occur.  In  the  example  above,  "enemy  attack"  and 
"no  attack"  are  obviously  an  exhaustive  set  of  outcomes. 

Other  exhaustive  sets  could  be  defined  in  this  instance,  for 
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example,  by  breaking  down  each  of  those  two  outcomes  to 
give  four  possibilities  in  ail; 

concerted  attack, 
skirmish, 

stand  fast  but  no  attack, 
retreat. 

The  choice  of  a particular  exhaustive  set  of  outcomes  is  an 
arbitrary  one,  depending  on  the  judgment  of  the  decision 
maker.  However,  no  important  possibilities  should  be  omitted. 
To  ensure  the  exhaustiveness  of  the  set,  the  decision  maker 
may  choose  to  include  a catch-all  outcome  labeled  "Other." 

Probabilities  Sum  to  1.00 

Once  a set  of  mutually  exclusive,  exhaustive  outcomes  is 
defined,  probabilities  are  determined  for  each  outcome,  and 
these  probabilities  will  all  be  In  the  range  from  0.00  to 
1.00.  The  last  important  feature  of  probabilities  to  be 
considered  is  that  the  total  of  all  these  probabilities  must 
sum  to  1.00.  For  example,  in  the  first  formulation  of  the 
military  prospects  illustration,  where  only  two  outcomes  are 
considered,  if  the  probability  of  "enemy  attack"  were  equal  to 
0.15,  then  the  probability  of  "no  attack"  would  have  to  be 
equal  to  0.85  to  make  their  sum,  0.15  + 0.85,  equal  to  1.00. 


Two  Types  of  Outcomes 


In  assessing  the  probabilities  of  the  different  outcomes 
of  an  event,  we  distinguish  between  two  cases:  where  there 
is  a limited  number  of  outcomes  and  where  there  is  a whole 
range  of  outcomes.  These  situations  require  probabilities 
that  are  called  discrete  probabilities  and  continuous  proba- 
bilities, respectively.  The  techniques  used  in  their  assess- 
ment are  somewhat  different. 

Discrete  Probabilities 


Both  the  examples  shown  in  Figure  3.1  are  set  up  for 
discrete  probabilities.  In  the  sales  volume  example,  only 
three  levels  of  sales  are  here  considered  as  possible  out- 
comes. The  probability  of  achieving  each  of  these  three 
levels  would  be  assessed,  and  their  total  value  would  be  1.00, 
since,  by  definition,  when  all  possible  outcomes  of  an  event 
have  been  considered,  their  probabilities  must  add  up  to  1.00. 
These  can  be  displayed  in  a probability  table  as  shown  below. 
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Sales  Level 
(Units  Sold) 

500 

1000 

1700 


Probability 

0.1 

0.7 

0.2 


Total:  1.0 


In  the  military  event  node,  only  two  outcomes  are  con- 
sidered possible,  attack  or  no  attack.  Two  probabilities 
would  be  assessed  and  their  total,  as  In  the  previous  example, 
would  bo  1.00.  If  the  analyst  thought  that  the  probability  of 
an  attack  were  1/100  or  0.01,  then  the  probability  of  "no 
attack"  would  have  to  be  1.00  - 0.01  or  .99.  These  values  are 
shown  In  the  table  below. 


Outcome  Probability 

Attack  0.01 

No  attack  0.99 

Total:  1.00 


Another  way  to  display  the  probabilities  Is  graphically, 
as  In  Figure  3-2. 

The  probability  for  any  outcome  is  read  simply  by  moving 
across  from  the  point  above  the  outcome  to  the  value  on  the 
vertical  scale. 


Continuous  Probabilities 


In  the  sales  volume  example,  we  know  that  considering 
only  three  sales  levels  is  really  a simplification  of  the 
problem.  The  sales  volume  could  take  on  any  of  a great  many 
values  In  a range  from,  say,  100  to  2500  units.  If  this  is 
the  case,  and  we  wish  to  know  the  probabilities  that  the 
outcome  will  be  anywhere  In  that  range,  then  we  require  a 
continuous  "probability  distribution"  over  that  range  rather 
than  discrete  probability  values.  The  result  of  assessing  a 
probability  distribution  is  shown  in  Figure  3-3.  In  this 
representation,  the  probability  of  any  smaller  range  of  values, 
such  as  sales  from  1000-2000  units,  is  found  by  measuring 
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PROBABILITY  DISTRIBUTION  FOR  PROJECTED  SALES  VOLUME 


I), 


the  area  under  the  curve  in  that  range.  The  total  area  under 
the  curve  la  equal  to  1.00. 

Another  way  to  represent  a probability  over  a range  Is 
to  draw  what  is  called  a "cumulative  probability  distribution." 
In  this  formulation,  shown  in  Figure  3-4,  the  quantity 
plotted  on  the  y (vertical)  axis  is  the  probability  that  the 
outcome  will  be  equal  to  the  corresponding  value  on  the  x 
(horizontal)  axis  or  leas.  For  instance,  in  the  example 
shown,  by  reading  up  from  the  sales  value  of  1000  units  to 
the  curve  and  across  to  the  cumulative  probability  scale 
(the  y axis),  we  see  that  the  probability  that  sales  will  be 


0 1000  2000  3000 


SALES 

(No.  of  Urtiti  SoW) 


Figure  3-4 

CUMULATIVE  PROBABILITY  DISTRIBUTION  FOR  PROJECTED  SALES  VOLUME 
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1000  units  or  less  is  equal  to  about  0.4.  The  cumulative 
probability  distribution  makes  it  easy  to  determine  the 
probability  that  sales  will  be  between  1000  and  2000  units. 
First,  the  cumulative  probabilities  up  to  those  values,  0.40 
and  0.92,  respectively,  are  read  off  the  graph.  Then  the 
smaller  value  is  subtracted  from  the  larger,  0.92-0.40-0.52, 
or  a 52  percent  chance  that  sales  will  be  between  1000  and 
2000  units. 

These  two  ways  of  showing  the  continuous  probability, 
the  probability  distribution  and  the  cumulative  probability 
distribution,  are  simply  alternative  ways  of  presenting  the 
same  information,  and  data  presented  in  one  form  can  be 
transformed  into  the  other.  For  some  purposes,  one  way  is 
more  convenient,  for  other  purposes  the  other.  It  is  easier 
to  see  the  probability  values  from  the  cumulative  curve 
since  they  are  read  directly  from  the  y axis,  but  most 
people  find  the  probability  distribution  in  Figure  3-3  more 
Intuitively  understandable.  Mathematically  speaking,  the 
cumulative  probability  distribution  is  the  integral  of  the 
probability  distribution. 

' When  we  assess  discrete  probabilities  we  are  estimating 
probabilities  for  each  of  the  outcomes)  when  we  assess  con- 
tinuous probabilities,  we  are  trying  to  establish  the  shape 
of  the  curve  over  the  different  outcomes. 


Quantitative  Sxpressions  of  Uncertainty 


We  cannot,  unfortunately,  directly  translate  semi- 
quantitative  expressions  of  uncertainty  such  as  "very  likely" 
into  probability  values.  The  reason  is  that  while  most 
people  would  agree  that  "very  likely"  means  a probability 
greater  than  0.5,  there  would  be  no  general  agreement  about 
how  much  more  than  0.5  the  probability  is  (0.8?  0.97  0.997). 
This  lack  of  agreement  in  the  use  of  such  terms  has  been 
shown  in  a number  of  experiments  (and  can  be  easily  verified 
by  the  reader  by  surveying  a few  friends) . 

For  example,  an  intelligence  analyst,  a professional  in 
the  art  of  reasoning  about  the  plausible,  was  asked  to  sub- 
stitute probability  estimates  for  some  of  the  verbal  quali- 
fiers in  an  article  he  had  written.  The  first  statement 
was:  "The  c<^a8e-fire  is  holding  but  it  could  be  broken 

within  a week."  The  analyst  said  that  he  meant  there  was  a 
30%  chance  the  cease-fire  would  be  broken  within  a week. 
Later,  an  analyst  who  had  helped  the  original  analyst  pre- 
pare the  statement  said  she  thought  that  there  was  an  80% 
chance  that  the  cease  fire  would  be  broken.  Yet,  both 
analysts  had  previously  believed  that  they  were  in  agreement 
about  what  could  happen. 
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A more  systematic  experiment  along  this  line  was  con- 
ducted by  NATO  intelligence  analysts  who  were  concerned  about 
the  problem  of  communication.  Several  different  sentences 
were  constructed  In  the  following  manner.  "It  Is  highly 
likely  that  the  Soviets  will  invade  Czechoslovakia,"  or  "It 
is  almost  certain  that  the  Soviets  will  invade  Czechoslovakia," 
or  "We  believe  that  the  Soviets  will  invade  Czechoslovakia." 

The  basic  structure  of  all  sentences  remained  constant;  only 
the  verbal  qualifiers  changed. 

Twenty-three  officers,  ranking  from  squadron  leader  to 
lieutenant  general,  served  in  the  experiment.  All  partici- 
pants were  familiar  with  reading  intelligence  publications. 

They  were  asked  to  indicate  the  probability  (in  percent)  that 
they  would  attribute  to  each  message  if  they  were  to  read  it 
in  that  form  in  an  intelligence  article.  The  results  of  that 
experiment  are  displayed  in  Figure  3-5.  Each  dot  represents 
one  reader's  probability  assignment.  While  there  were  some 
phrases,  particularly  "better  than  even,"  which  were  inter- 
preted in  much  the  same  way  by  most  of  the  readers,  it  can  be 
seen  that,  in  general,  there  was  tremendous  variation. 

In  1964,  Sherman  Kent  attempted  to  mitigate  this  problem 
within  the  intelligence  community  by  proposing  a scale  range 
of  probabilities  for  various  qualifying  phrases.  These 
ranges  are  indicated  by  the  shaded  areas  in  Figure  3-5. 

Clearly,  the  readers  in  this  experiment  were  not  using  the 
Sherman  Kent  scale  even  though  they  were  familiar  with  it. 

Even  if  the  officers  had  been  using  the  scale,  the  wide 
ranges  for  somu  ol  tho.  wUrunnH  woipy  still  permit  considerable 
variation  in  interpretation. 

This  and  other  demonstrations  provide  strong  evidence 
that  significant  miscommunication  can  and  does  occur  among 
analysts.  Seeming  agreement  among  analysts  on  an' item  being 
discussed  could,  therefore,  be  an  artifact  induced  by  the 
impreciseness  of  the  verbal  qualifiers.  In  reality,  there 
could  be  considerable  disagreement. 

The  fact  that  verbal  qualifiers  are  imprecise  descrip- 
tions of  levels  of  certainty  provides  strong  motivation  for 
the  use  of  a quantitative  language  of  uncertainty.  Appro- 
priate use  of  numbers  allows  uncertainty  to  be  expressed  with 
precision.  If  a manager  uses  numbers,  either  percentages  or 
odds,  to  convey  a degree  of  belief  about  the  likelihood  of 
future  events,  and  the  numbers  are  carefully  chosen  to  reflect 
the  manager's  uncertainty,  then  these  numbers  can  be  easily 
converted  into  probabilities. 
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Figure  3-5 

WHAT  UNCERTAINTY  STATEMENTS  MEAN  TO  DIFFERENT  READERS 
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1 

i 


Percentages 

I 

Percentages  translate  directly  Into  probabilities 
simply  by  changing  the  scale  for  percentages  running  from  0 i 

through  100  to  a probability  scale  going  from  0 through  | 

1.00.  Translation  is  easily  accomplished  by  dividing  the 
percentage  figure  by  100.  For  example,  if  a manager  be- 
lieves that  there  la  an  "80-percent  chance"  that  the  com- 
petition will  respond  to  his  new  advertising  campaign  by 
increasing  their  own  advertising  expenditure,  then  he  may  be 
understood  to  mean  that  the  probability  of  such  a response 
is  0.80  (80  percent).  We  may  then  show  those  probabilities 
on  the  relevant  branches  of  a fork  in  a decision  tree,  as  in 
Figure  3-6. 
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prob-O.W 


Competitor  increases  advertising 


Competitor  does 
not  Increase  advertising 


Figure  3-6 

PROBABILITIES  ON  THE  BRANCHES  OF  AN  EVENT  FORK 


Odds 

Some  managers  are  accustomed  to  quoting  odds  to  express 
their  feelings  of  uncertainty.  These  values  can  also  be 
directly  converted  to  probabilities.  Suppose,  for  instance, 
a Civil  War  military  analyst  had  said  before  the  battle  at 
Gettysburg  that  the  odds  were  3 to  1 in  favor  of  a victory 
by  the  Union  troops.  This  statement  of  odds  is  equivalent 
to  saying  the  probability  of  victory  is  0.75.  In  other 
words,  3-to-l  odds  implies  that  the  probability  of  winning 
must  be  three  times  the  probability  of  losing.  Furthermore, 
the  total  of  the  probabilities  of  winning  and  losing  must 
sum  to  1.00.  The  only  numbers  between  0 and  1 satisfying 
these  two  conditions  are  0.75  and  0.25.  We  can  see  that  an 
odds  statement  is  an  expression  of  the  ratio  of  the  proba- 
bilities of  the  two  outcomes  being  considered.  In  this 
example,  the  ratio  is  0.75/0.25  or  3/1  or  3 to  1.  Alge- 
braically, the  rule  for  converting  odds  to  probabilities 
where  there  are  two  outcomes  can  be  stated  as  follows: 
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If  the  odds  are  M to  N, 
bilities  are: 


then  the  corresponding  proba- 


M 

Fmj 


+ 


N 

M + N 


- 1.00. 


In  the  example  above  this  becomes: 

^ I Y (i.e.,  3/4  or  .75)  + 3-^  (i.e.,  1/4  or  .25)  - 1.00 

Another  common  way  of  discussing  uncertainty  is  to  say 
that  the  chances  are  so  many  "in"  or  "out  of"  some  total.  For 
exeunplef  a manager  may  say  "the  chances  are  1 In  100"  of 
something  occurring,  or  in  another  instance,  "y~out  of  10" 
that  a particular  outcome  will  occur.  Rather  than  adding  up 
the  two  figures  in  the  statement  to  get  the  denominator  for 
the  probability,  as  was  the  case  for  the  "odds"  expressions, 
the  second  value  is  taken  directly  to  be  the  denominator,  as 


shown  below: 

Converts 

Chances 

to 

Probability 

1 in  100 

1/100 

9 out  of  10 

9/10 

Probability  Assessment  Techniques 


When  uncertainties  are  not  stated  as  odds  or  percentages, 
some  more  complicated,  laborious  procedures  must  be  used  to 
elicit  feelings  of  uncertainty  in  numerical  terms.  A great 
deal  of  research  in  this  area  has  been  carried  out  to  develop 
and  test  various  probability  assessment  techniques.  Among  the 
more  frequently  used  methods  are: 

Probability  Wheels, 

Reference  Lotteries, 

Relative  Probability, 

Quartile  Assessment, 

Trisection, 

Other  Fractlle  Assessment, 

Cumulative  Distribution  Assessment. 

The  first  three  methods  are  used  to  assoss  discrete 
probabilities;  the  others  are  techniques  for  assessing  oon-> 
tinuous  probabilities.  The  operation  of  the  techniques  is 
described  below. 
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Probability  Wheel 


The  probability  wheel  measures  probabilities  for  people 
who  are  uncomfortable  (usually  because  they  object  to  the 
implication  of  precision)  with  assigning  numbers  to  uncer- 
tainty. The  wheel  consists  of  a disk  shaded  in  two  colors, 
with  a spinning  pointer.  The  colors,  blue  and  orange,  can 
be  adjusted  so  that  the  orange  portion  goes  from  being  a 
tiny  sliver,  1 percent,  all  the  way  up  to  99  percent  of  the 
disk.  The  probability  assessor  who  is  trying  to  assess  his 
feelings  about,  for  example,  the  probability  that  the  mon- 
archy in  Spain  will  be  overthrown  next  year  is  asked  to 
adjust  the  amount  of  orange  (either  decreasing  or  increasing 
it)  until  the  wheel  is  configured  so  that  the  assessor 
believes  that  it  is  as  likely  that  the  monarchy  in  Spain 
will  be  overthrown  as  that  the  pointer  on  the  wheel  when 
spun  will  stop  on  the  orange  color.  The  assessor  then  looks 
at  the  back  of  the  wheel  where  a scale  shows  what  proportion 
of  the  wheel  is  orange.  This  value  is  taken  to  be  the 
probability  that  the  wheel  will  land  on  orange  and  is  taken 
to  be  the  assessor's  probability  that  the  Spanish  monarchy 
will  be  overthrown. 


Reference  Lottery 

The  reference  lottery  is  a technique  that  helps  an 
assessor  to  focus  fairly  narrowly  on  his  feelings  about  a 
discrete  probability  value,  particularly  when  only  two 
possible  outcomes  of  the  event  are  being  considered.  Suppose, 
for  example,  that  a political  analyst  would  like  to  estimate 
quantitatively  the  probability  that  a Republican  candidate 
will  be  the  winner  of  the  next  U.S.  presidential  election, 
as  shown  in  Figure  3-7A.  An  estimate  of  that  probability 
can  be  made  by  considering  the  following  imaginary  situation: 
First,  the  assessor  is  asked  to  think  of  a prize  that  he 
would  like  to  have,  such  as  a trip  to  Greece,  series  tickets 
for  the  local  sports  arena,  a Picasso  etching,  and  so  on. 

Then  a lottery  is  hypothesized  where  a single  ball  will  be 
drawn  blindly  from  an  urn  containing  100  balls,  some  red  and 
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PROBABILITY 


OUTCOMES 


RepubHcpn  wins 


Republican  loses 


B.  Rafaranc*  Lottary 


PROBABILITY 


OUTCOMES 


Red  ball  is  drawn; 
player  gets  prize 


Green  ball  is  drawn; 
player  gets  no  prize 


Figure  3-7 

AN  UNCERTAIN  EVENT  AND  A REFERENCE  LOTTERY 
USED  TO  ASSESS  THE  PROBABILITY  OF  THE  UNCERTAIN  EVENT 


some  green.  The  drawing  will  be  made  as  soon  as  the  winner 
of  the  eleotion  is  known,  but  not  before.  Now  suppose  there 
are  60  red  balls  in  the  urn.  The  assessor  is  then  asked  if 
he  would  rather  have  the  opportunity  of  letting  his  prize  be 
contingent  on  drawing  a red  ball  in  the  reference  lottery  or 
contingent  on  having  a Republican  winning  the  presidential 
election.  If  the  assessor  selects  the  reference  lottery, 
where  the  probability  of  getting  the  prize  is  60/100  or  0.6, 
then  we  conclude  that  his  assessment  of  the  probability  of  a 
Republican  election  winner  is  less  than  0.6.  By  changing 
the  composition  of  the  lottery  urn  so  that  it  contains,  say, 
40  red  balls  out  of  100,  we  can  find  out  if  he  still  prefers 
the  lottery  to  the  election  outcome.  If  so,  his  assessment 


of  thf:  probability  of  a Republican  win  is  less  than  0.4.  We 
continue  to  change  the  composition  of  the  urn  until  we  have 
reached  one  such  that  the  assessor  is  indifferent  between 
getting  the  prize  by  winning  the  lottery  or  by  having  a 
Republican  winner  in  the  presidential  election..  The  proba- 
bility of  winning  the  lottery  (taken  directly  from  the 
composition  of  the  urn)  is  then  equal  to  the  assessor's 
personal  probability  or  degree  of  belief  that  a Republican 
will  win  the  election 

Rcilative  Likelihood 

When  the  discrete  probabilities  of  an  event  which  has 
several  outcomes  are  considered,  it  is  often  helpful  to  ask 
the  assessor  to  make  a number  of  judgments  about  the  rela- 
tive likelihood  between  pairs  of  outcomes.  In  the  simpli- 
fied version  of  the  "sales  level"  example  where  three  out- 
comes (500  units,  1000  units,  and  1700  units)  were  specified, 
the  assessor  could  be  asked  the  following- questions : 

How  much  more  likely  is  it  that  sales  will  be  1000 

units  rather  than  500?  Twice  as  likely?  Five  times  as 

likely?  Less  likely? 

What  about  the  likelihood  of  1000  units  compared  to 

1700  units? 

And  500  units  compared  to  1700  units? 

After  these  relative  relationships  are  elicited  and  then 
discussed  and  reassessed  to  eliminate  any  inconsistencies,^ 
the  values  obtained  (which  are  actually  a form  of  odds)  are 
converted  to  probabilities. 

Quartile  Assessments 


The  foregoing  methods  are  designed  to  assess  discrete 
probabilities.  In  the  case  of  continuous  probabilities, 
however,  we  need  to  develop  an  entire  probability  distribu- 
tion or  curve.  Most  techniques  for  assessing  probability 
distributions  consist  of  ways  of  obtaining  a few  points  on 
the  curve  and  then  r.ketching  in  an  approximation  to  the  rest 
of  the  curve.  The  general  name  for  one  group  of  these 
techniques  is  "fractile  assessment,"  and  depending  on  the 
points  beii'.g  estimated,  the  specific  techniques  are  called 


2 

It  would  be  inconsistent,  for  example,  to  say  that  A is 
twice  as  likely  as  B,  B is  three  times  as  likely  as  C,  and 
C is  half  as  likely  as  A,  since  the  latter  would  imply  that 
A is  twice  as  likely  as  C,  which,  by  the  first  statement, 
makes  B and  C equally  likely,  thus  contradicting  the  second 
statement. 
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quartile  assessment,  trisectlon,  evtremes  assessment,  and  so 
on.  How  the  probability  information  is  elicited  to  yet  these 
points  is  most  easily  shown  by  an  example. 

Consider  the  case  of  a manager  responsible  for  a research 
group  who  wants  to  assess  a probability  distribution  for  what 
dollar  amount  will  be  allocated  to  U&D  in  the  next  budget 
period.  In  other  words,  he  would  like  to  quantify  the  like- 
lihocd  of  various  different  possible  budget  amounts  in  some 
range.  Suppose  the  range  of  possible  values  is  from  seven 
million  dollars  to  fifteen  million  dollars.  One  way  to  elicit 
this  probability  information  is  by  assessing  three  quartiles 
(Ql»  U2»  U3)  which  are  points  that  divide  the  range  into  four 
equally  likely  parts  (or  quarters) . Notice  the  emphasis  on 
the  words  ‘'equally  likely."  These  points  will  not  normally 
divide  the  range  into  equally  wide  parts  (see  Figure  3-8A) , 
but  rather  into  segments  such  that  the  actual  budget  amount  is 
just  as  likely  to  be  between  the  minimum  and  point  Qj,  or 
between  Qi  and  02*  between  .02  03/  Of  between  Q3  and 

the.  maximum. 

Remember  that  for  a continuous  probability  distribution 
tho  probability  for  any  range  of  values  is  found  by  measuring 
the  area  under  the  curve  between  end  points  of  the  range. 

Since  the  quartiles  divide  the  whole  range  into  four  equally 
likely  parts  and  the  total  probability  is  1.00  or  100%,  the 
probability  for  each  of  the  segments  must  be  0.25  or  25%. 
Information  about  what  the  quartiles  are,  therefore,  is 
sufficient  to  allow  us  to  sketch  in  (either  by  eye  or  by  using 
a computer  program)  the  approximate  shape  of  the  probability 
curve  (see  Figure  3-8B) . The  closer  together  the  quartiles 
are,  the  more  peaked  the  curve  will  be  (Figure  3-8C) . When 
the  assessed  quartiles  are  far  apart,  the  curve  will  be  flatter 
(Figure  3--8D)  . The  location  of  these  quartiles  is  determined 
by  asking  the  decision  maker  to  respond  to  questions  like  the 
following ; 

(1)  What  is  the  value  such  that  you  think  the  R&D  budget 
is  just  as  likely  to  be  above  as  to  be  below  this 
figure?  (This  number  is  02,  the  second  quartile 
because  it  div.ides  the  range  into  two  equally  likely 
parts,  so  each  of  these  parts  will  have  a proba- 
bility of  0.5  or  50%.  This  is  the  probabilistic 
midpoint  of  the  range.) 

Suppose  the  decision  maker  has  given  "$10  million"  as  his 
estimate  of  Q2.  He  is  then  asked: 

(2)  Now  conside:.:  values  that  are  less  than  $10  million. 
What  single  value  divides  this  part  of  the  range 
(the  minimum  to  $10  million)  into  two  equally 
likely  parts?  (This  new  value  will  be  Qi-) 
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B.  Equivalent  Curve  for  Ponible  Quartilo  Valuai 
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R&D  BUDGET  ($  In  Mllllont) 

C.  Peaked  Curve 


R&D  BUDGET  ($  in  Millioni) 

D.  Fiat  Curve 


R&D  BUDGET  ($  in  Millioni) 


Figure  3-8 

QUARTILE  ASSESSMENTS 


The  value  of  Q3  is  found  in  a similar  manner  by  looking  at 

the  range  from  Q2  to  the  maximum.  In  the  example  shown  in 

Figure  3-8C,  these  three  quartile  values  assessed  turned  out 
to  be  approximately; 

Q2  “ $10,8  million, 

Ql  = $10.2  million, 

Q3  " $11.5  million. 

Trisection 


An  alternative  to  assessing  the  quartiles  of  a proba- 
bility distribution  is  to  ask  the  decision  maker  to  estimate 
the  two  points  and  T2)  that  divide  the  range  into  three 
equally  likely  parts.  This  process  is  called  " tri8ection7" 
and  the  resulting  segments  of  the  range  each  have  probabili- 
ties equal  to  33  1/3  percent  of  containing  the  actual  value 
of  the  variable  whose  probability  is  assessed,  in  this 
example,  the  budget  (see  Figure  3-9). 


Figure  3-9 
TRISECTION 


Other  Fractile  Assessment 


More  than  two  or  three  points  on  the  curve  can  be 
assessed  if  necessary  for  the  problem  being  studied.  The 
deciding  factors  here  are  how  much  detail  or  accuracy  in  the 
curve  is  desired  and  how  much  time  and  effort  are  to  be 
spent  on  the  probability  assessment  portion  of  the  analysis. 
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For  some  problems,  information  about  the  extremes  of  the 
distribution  may  be  relevant  (in  the  budget  example,  for 
instance,  an  extreme  would  be  the  value  such  that  there  is 
only  one  chance  in  100  of  the  budget  being  less  than  this 
value).  There  are  special  methods,  not  discussed  here,  that 
can  be  used  tc  elicit  this  Information. 


Cumulative  Distribution  Assessment 


A probability  distribution  can  also  be  obtained  by 
asking  for  estimates  of  points  on  the  cumulative  probability 
curve.  Since  the  cumulative  probability  distribution  is  a 
graph  of  possible  values  of  an  uncertain  quantity  vs  the 
probability  of  the  occurrence  of  that  value  or  something  less, 
the  questions  asked  to  elicit  points  on  the  curve  would  be 
like  thoue  shown  below.  For  the  R&D  budget  example: 

(1)  What  is  the  value  such  that  you  think  there  is  a 
0.5  probability  that  the  budget  will  be  this  value 
or  less? 

(2)  What  is  the  value  such  that  you  think  there  is  a 
0.25  probability  that  the  actual  budget  will  turn 
out  to  be  less  than  this? 

(3)  What  is  the  value  such  that  you  think  there  is  a 
0.75  probability  that  the  actual  budget  will  be 
less  than  this? 

Notice  that  the  values  elicited  in  this  manner  should  be  the 
same  as  the  three  quartiles  (Q2#  Ql>  Q3)  assessed  for  the 
probability  distribution.  This  procedure  is  simply  another 
way  of  getting  the  same  information  about  the  assessor's 
uncertainty.  For  example,  the  value  such  that  the  assessor 
thinks  there  is  a 0.5  probability  that  the  budget  will  be 
this  value  or  less  (question  1 here)  should  be  the  same  as 
the  value  such  that  the  assessor  thinks  the  R&D  budget  is 
just  as  likely  to  be  above  as  to  be  below  this  figure  (ques- 
tion 1 under  Quartile  Assessment) . Figure  3-10  shows  the 
probability  distributions  B,  C,  and  D of  Figure  3-8  redrawn 
in  cumulative  form.  A comparison  of  these  two  figures  shows 
that  the  more  peaked  the  probability  distribution  is,  the 
steeper  will  be  the  slope  of  thf}  cumulative  distribution 
curve.  Despite  the  fact  that  the  two  types  of  curves  are 
based  on  and  display  the  same  information,  most  people  are 
more  comfortable  working  with  the  probability  information  in 
the  first  form. 
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A.  Equivalent  Curve  for  Potiible  Quartile  Values 


B.  Peaked  Curve  ' 


RltD  BUDGET  (B  in  millloni) 


C.  Flat  Curve 


R&D  BUDGET  ($  in  millioni) 


Figure  3-10 

ASSESSED  CUMULATIVE  PROBABILITIES 
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Values  of  the  Variable 

In  the  methods  discussed  so  far,  the  assessors  are 
given  specific  probability  statements  and  asked  to  estimate 
the  value  of  the  uncertain  quantity  which  corresponds  to  the 
specified  probability  (e.g.,  "What  is  the  value  of  the 
variable  such  that  you  think  there  is  a 0.75  probability 
that  the  actual  budget  will  be  less  than  this?").  An  alter- 
native way  to  elicit  the  assessor's  personal  probabilities 
is  to  reverse  the  question  by  specifying  the  value  or  range 
of  the  variable  and  asking  the  assessor  for  a numerical 
estimate  of  the  corresponding  probability  (e.g.,  "What  do 
you  think  is  the  probability  that  the  budget  will  be  $11 
million  or  less?"). 

Standard  Mathematical  Forms 


The  assessment  of  probability  distributions  can  be 
simplified  in  some  cases  by  assuming  that  the  probability 
curve  has  some  specific  mathematical  form.  That  is,  the 
curve  can  be  written  according  to  a mathematical  equation. 

When  some  points  on  the  curve  have  been  assessed,  one  can 
solve  for  the  values  of  the  unknowns  in  the  equation  (called 
the  "parameters")  and  calculate  mathematically  the  remaining 
probabilities  of  any  points  or  ranges.  The  accuracy  of  the 
probabilities  obtained  in  this  way  depends  upon  how  good  the 
Initial  assessments  are  and  upon  how  realistic  the  assump- 
tion is  that  the  desired  distribution  follows  the  mathe- 
matical form  chosen. 

Two  examples  of  commonly  used  mathematical  forms  are 
the  bell-shaped  curve,  known  as  the  "normal"  or  "Gaussian" 
distribution,  and  the  beta  distribution.  Typically,  these 
distributions  are  assessed  by  specifying  either  two  quar- 
tiles  (the  first  and  third)  or  the  mean  and  variance  of  the 
distribution.  If  an  interactive  computer  is  used  as  an  aid, 
the  entire  distribution  can  be  computed  immediately,  and  any 
selected  ranges  printed  out. 

Comparing  Methods 

Since  the  probability  assessments  obtained  by  different 
means  often  differ  among  themselves  to  varying  degrees, 
depending  upon  which  assessment  methods  are  used,  it  is  a 
good  idea  to  assess  the  same  probability  in  more  than  one 
way  and  then  compare  the  values  obtained.  If  they  differ, 
some  additional  thought  and  discussion  will  usually  enable 
the  assessor  to  settle  on  a single  value  in  which  the  assessor 
has  more  confidence  than  any  of  the  initial  values. 

For  example,  a distribution  assessed  by  trisection  can 
be  refined  by  means  of  the  quartile  technique.  The  two 
curves  obtained  this  way  are  reconciled  and  then  further 


79 


tested  by  asking  for  probabilities  associated  with  specific 
values  of  the  variable  to  see  whether  these  probabilities 
agree  with  those  implied  by  the  curve. 

When  time  for  eliciting  probabilities  by  several  pro- 
cedures and  for  comparing  results  is  limited,  however,  we 
must  consider  which  one  assessment  procedure  can  be  follov;ed. 
The  answer  depends  not  only  on  the  task  (the  initial  decision 
in  a tree  may  be  so  insensitive  to  the  shape  of  the  proba- 
bility distribution  that  a very  rough  sketch  based  on  tri- 
seotion  may  be  adequate) , but  also  on  the  assessor.  If  he 
is  unfamiliar  with  probabilities  and  reluctant  to  make 
numerical  judgments  of  probability,  then  quartile  or  tri- 
section assessment  methods  should  be  used.  If  he  is  com- 
fortable with  probability  assessment,  then  the  method  of 
specifying  values  and  asking  for  assessments  of  cumulative 
probability  results  in  a distribution  that  is  usually  less 
biased  than  the  distribution  that  results  from  quartile  or 
trisection  assessment.  If  the  extreme  ends  of  the  range  of 
the  distribution  are  important  to  the  decision,  then  inter- 
active assessment  with  a computer^  which  uses  standard 
mathematical  functions  allows  their  careful  examination. 


Updating  Assessments 


Probability  assessments  are  designed  to  express  quantita- 
tively the  assessor's  current  feelings  of  uncertainty  about 
some  outcome.  In  other  words,  they  reflect  the  assessor's 
present  state  of  knowledge.'  As  the  assessor  gains  additional 
information  about  the  uncertain  quantity  or  event,  the 
probabilities  may  change.  For  example,  suppose  an  intelli- 
gence analyst  has,  after  careful  consideration  of  many 
factors,  assessed  the  probability  of  a coup  in  Freedonia  as 
being  equal  to  0.60.  Then,  after  making  this  estimate,  the 
analyst  learns  that  the  children  of  the  Freedonian  President 
are  traveling  to  London  for  a short  vacation.  The  analyst 
would  like  to  incorporate  this  new  data  into  the  probability 
estimate.  It  is  easy  to  tell  in  a general  way  how  such 
information  would  alter  the  probabilities.  In  the  case  of 
an  event  like  the  one  above  having  two  possible  voutcomes,  it 
is  natural  to  expect  that  if  the  additional  information  is 
in  greater  accord  with  the  likelihood  of  one  outcome  than 
with  that  of  the  other,  then  the  probability  of  the  one  will 
be  increased,  and  vice  versa. 


3 

An  extensive  suite  of  programs  has  been  developed  at 
Harvard  University.  See  Schlaifer,  Robert,  Computer  Pro- 
grams for  Elementary  Decision  Analysis.  (Cambridge,  MA: 
Harvard  Univ.  Press , 1 7 1 ) . 
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To  compute  precisely  the  appropriate  amount  of  change 
in  the  probability  is  a more  complicated  matter.  An  early 
and  useful  development  In  the  theory  of  probability  was 
Thomas  Bayes'  proposal  of  a formal^  mathematical  procedure 
for  incorporating  new  information  to  revise  a probability 
estimate.  This  procedure,  called  "Bayes*  Theorem,"  is 
explained  in  detail  and  illustrated  in  Appendix  A,  and  the 
application  of  that  theorem  in  cases  having  multiple  value 
dimensions  is  presented  in  Appendix  B.  In  the  case  of 
discrete  probabilities,  Bayes*  Theorem  can  easily  be  applied 
to  revise  the  originally  assessed  probabilities,  provided 
the  new  information  is  expressed  in,  or  can  be  converted  to, 
numerical  form.  For  continuous  uncertain  quantities,  on  the 
other  hand,  the  degree  of  difficulty  in  applying  Bayes' 
Theorem  depends  upon  the  particular  mathematical  formulation 
of  the  curve. 

When  using  Bayes'  Theorem,  the  initially  assessed 
probability  is  called  a "prior"  probability  and  the  revised 
or  updated  probability  is  called  a "posterior"  probability 
(meaning  that  it  is  the  revised  probability  after  the  new 
information  is  received).  The  form  in  which  the  new  data  is 
expressed  is  called  a "likelihood  ratio."  In  the  example 
cited  above,  the  analyst  would  have  to  express  his  feelings 
about  the  implications  of  the  new  data  in  this  way: 

"What  is  the  likelihood  that  the  President's  family 
would  leave  the  country  if  a coup  were  imminent?" 

and  to  determine  the  relevance  of  this  new  information  to 
the  other  possibility, 

"What  is  the  likelihood  that  the  President's  family 
would  travel  abroad  in  ordinary  times?" 


Bias  in  Assessments 


A probedaility  assessor  must  devote  a certain  amount  of 
time  and  effort  to  learning  how  to  use  the  appropriate 
assessment  technique  or  techniques  for  his  problems  before 
useful  probcdDility  values  can  be  obtained.  Even  after  the 
mechanics  of  the  method  are  understood,  however,  research 
has  shown  that  assessors  typically  are  prey  to  a number  of 
systematic  errors  or  biases  in  making  their  assessments. 
Accordingly,  to  obtain  the  best  assessments,  some  means  must 
be  developed  for  eliminating  or  correcting  these  biases. 
Ideally,  the  probability  assessment  should  reflect  as  closely 
as  possible  an  assessor's  true  feelings  of  uncertainty. 

Events  given  high  probability  values  should  occur  frequently 
and  those  with  low  probability  values  less  frequently.  One 
way  to  check  assessments  for  this  characteristic  is  to 
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collect  a large  number  of  assessments  from  the  same  assessor 
and  see  whether  the  assessed  probability  corresponds  to  the 
frequency  of  occurrence.  For  example,  consider  the  case  of  a 
weather  forecaster  who  has  assessed  precipitation  probabilities 
for  several  months;  looking  back  over  his  records,  we  would 
expect  that  if  the  assessments  are  good  estimates  of  the 
forecaster's  uncertainty,  then  it  would  have  rained  on  about 
60%  of  the  days  for  which  the  forecaster  said  the  probability 
of  rain  was  0.6.  A probability  assessor  who  exhibits  this 
characteristic  is  said  to  be  a "good"  assessor  or  "well  cali- 
brated. " 

Assessors  who  are  not  well  calibrated,  on  the  other  hand, 
may  exhibit  any  of  several  types  of  biases.  For  example,  they 
may  be  overconfident,  that  is,  too  sure  of  themselves  and 
consequently  assess  probabilities  that  are  always  too  extreme 
(too  close  to  1.00  or  0.00)  for  discrete  events,  or  assess 
curves  that  are  too  narrow  or  tight  for  continuous  proba- 
bilities. Some  assessors,  on  the  other  hand,  may  be  system- 
atically under confident  in  their  assessments.  Another  bias 
observed  is  the  phenomenon  of  availability,  the  tendency  of 
people  to  assess  probabilities  according  to  the  number  of 
instances  of  the  event  that  they  remember  and  how  easily  they 
recall  them.  These  factors  may  or  may  not  be  related  to 
correct  probability  of  occurrence.  For  example,  the  letter  k 
is  much  more  likely  to  occur  as  the  third  letter  of  a word  in 
the  English  language  than  as  the  first  letter,  yet  most  people 
judge  it  as  more  likely  to  be  a first  letter.  Presumably, 
this  is  because  in  making  such  a judgment,  people  try  to  think 
of  words  either  starting  with  k or  having  k as  a third  letter. 
It  is  easier  to  think  of  words  beginning  with  k,  and,  there- 
fore, they  are  judged  to  be  more  frequent. 

There  are  two  ways  of  overcoming  the  bias  in  probability 
assessments.  One  is  to  observe  an  assessor  long  enough  to 
discover  which  particular  biases  he  exhibits  and  to  what 
degree.  Then  a correction  factor  is  computed  and  applied  to 
any  future  assessments  to  eliminate  the  bias.  The  more  usual 
practice,  however,  is  to  explain  the  kinds  of  bias  and  to 
train  the  assessor  to  avoid  them  so  that  he  becomes  well 
calibrated.  This  training  consists  of  repeated  assessments 
with  feedback  presented  in  the  form  of  a "score"  that  tells 
the  assessor  how  well  he  is  doing  compared  to  a perfectly 
calibrated  assessor.  Appendix  C presents  a discussion  of 
scoring  rules  for  training  probability  assessors. 

Bias  also  appears  when  people  are  asked  judginentally  to 
revise  an  initial  probability  estimate  on  the  basis  of  new 
information.  This  bias  is  eliminated,  however,  by  following 
the  formal  updating  procedure  of  Bayes*  Theorem. 
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CHAPTER  4 


PROBABILITY  DIAGRAMS  AND  HIERARCHICAL  INFERENCE 


When  determining  the  probability  of  an  event  that  is 
related  to  other  events,  an  assessor  will  often  find  that  it 
is  easier  to  judge  likelihoods  for  the  related  events  than 
it  is  to  assess  the  probability  of  the  event  of  interest. 
Probability  diagrams  provide  one  of  the  most  useful  and 
straightforward  techniques  for  decomposing  the  problem  into 
its  parts  and  assessing  those  probabilities.  Instead  of 
trying  to  assess  the  overall  probability  of  an  event  or  out- 
come, the  analyst  can  use  probability  diagrams  to  assess  the 
chance  of  that  event  by  first  assuming  some  of  those  related 
events  have  occurred  and  then  by  assuming  those  related 
events  have  not  occurred. 

Consider  the  following  problems  in  which  it  is  more 
effective  to  assess  the  components  than  it  is  to  assess  the 
total  problem: 

1.  The  probability  of  rain  tomorrow  will  depend  on 
several  conditions,  barometric  pressure  and  pre- 
vailing winds. 

2.  The  probability  of  new  international  monetary 
reform  taking  place  this  year  will  depend  mainly 
on  whether  or  not  countries  A and  B both  agree  to 
support  the  proposed  changes. 

3.  The  probability  of  a coup  in  a particular  country 
may  depend  jointly  on  three  factors:  the  state  of 
the  economy,  the  country's  support  of  a particular 
general,  and  threat  of  aggression  from  a neighbor- 
ing country. 

4.  An  analyst  is  interested  in  determining  the  proba- 
bility that  country  X will  launch  a major  attack 
across  the  border  of  country  Y.  He  has  come  to 
the  conclusion  that  X is  more  likely  to  attack  if 
the  leader  of  Y dies  and  if  there  is  political 
upheaval. 

Consider  problem  #1,  determining  the  probability  of  rain 
occurring  tomorrow  morning.  Since  the  possibility  of  rain 
will  be  dependent  on  several  uncertain  conditions  like  barometric 
pressure  and  prevailing  winds,  it  will  be  difficult  to 
assess  the  overall  probability  by  looking  at  the  conditions 
from  a global  viewpoint.  A simpler  method  is  to  decompose 
the  problem  into  separate  components  and  to  assess  the 
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implications  of  each  section  individually.  Then  it  is 
possible  to  reconstruct  the  problem  by  applying  the  laws 
of  probability  to  the  decomposed  assessments.  The  key  is  to 
attack  tho  problem  from  its  Integral  parts  and  to  rebuild 
rather  than  to  estimate  an  overall  probability  from  a global 
viewpoint. 

Structure 

Once  the  analyst  has  decomposed  the  problem  into  those 
components  he  considers  relevant,  he  can  use  a probability 
diagram  to  structure  the  problem.  For  example,  if  our 
weatherman  were  concerned  with  the  probability  of  rain,  he 
could  structure  a simple  probability  diagram  (see  Figure  4-1) 
with  two  possible  outcomes,  or  target  variables,  that  is,  rain 
or  no  rain,  with  probabilities  Pi  and  P2,  respectively. 


RAIN 


Figure  4-1 

SIMPLE  DIAGRAM 

If  the  weatherman  were  now  asked  to  assess  the  overall 
probabilities.  Pi  and  P2,  without  knowing  what  other  events 
will  occur,  he  will  have  difficulty  making  the  assessment. 
If  he  feels  that  the  probability  of  rain  will  increase  or 
decrease  depending  on  whether  the  winds  will  be  from  the  NW 
or  from  the  SW,  ho  can  restructure  his  probability  diagram 
as  in  Figure  4-2. 


RAIN 
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Notice  that  thf  far  right-hand  branches  of  Figure  4-2  still 
have  the  seune  target  variables,  rain  and  no  rain,  as  do  the 
branches  in  Figure  4-1.  He  is  still  concerned  with  the  same 
outcome;  only  now  he  bases  his  estimate  of  the  likelihood  of 
that  outcome  directly  on  wind  condition.  In  order  to  assess 
the  likelihood  of  rain  for  Pq,  he  can  first  assume  that  the 
winds  will  be  from  the  NW,  and  assuming  NW  winds,  he  can 
more  easily  determine  the  probability  of  rain.  Notice  that 
it  would  be  much  easier  to  assess  Pc  in  Figure  4-2  than  it 
would  be  to  assess  Pi  in  Figure  4-1. 

In  a similar  manner,  if  the  weatherman  feels  wind 
direction  will  be  influenced  by  the  existence  of  a low-  or 
high-pressure  area  and  could  restructure  his  probability 
diagram  accordingly,  the  result  would  appear  as  in  Figure  4-3. 


RAIN 


If  barometric  pressure  and  wind  direction  were  the  only 
conditions  affecting  the  likelihood  of  rain,  then  Figure 
4-3  would  be  a complete  probability  diagram  for  this 
problem. 
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The  overall  diagram  of  this  problei.i  has  been  constructed 
in  the  shape  of  a tree  with  each  of  the  possible  sets  of 
outcomes  represented  as  a separate  branch  of  the  tree.  Pro- 
gressing to  the  right,  an  additional  assumption  has  been 
made  each  time  a point  is  reached  from  which  branches 
emerge,  this  point  being  called  a node.  For  example,  if  the 
analyst  follows  the  numbered  branches  of  Figure  4-3  at  the 
upper  branch  of  node  #1,  he  assumes  that  a low-pressure  area 
exists.  At  the  lower  branch  of  node  #2,  he  assumes  that  in 
addition  to  a low-pressure  area,  the  winds  are  from  the  SW. 
Finally,  from  the  upper  branch  of  node  #3,  he  assumes  that  the 
likelihood  of  rain  is  determined  by  both  a low-pressure  area 
and  SW  winds.  Notice  that  for  this  branch  it  would  be  much 
easier  to  assess  the  likelihood  of  rain  tomorrow  by  first 
assuming  a low-pressure  area  and  SW  winds  than  it  would  be 
to  estimate  the  likelihood  of  rain  tomorrow  without  making 
any  previous  assumptions.  The  same  is  true  for  each  of  the 
other  branches.  We  call  probabilities  arrived  at  by  combining 
probabilities  estimated  for  relevant  components  conditional 
probabilities. 

Once  the  probabilities  have  been  estimated  for  each  of 
the  eight  branches,  the  weatherman  can  total  the  probabilities 
for  those  branches  resulting  in  rain  and  determine  the 
original  overall  target  variables,  and  P2.  We  turn  next 
to  the  rules  that  must  bo  followed  in  constructing  probability 
diagrams  and  assessing  probabilities?  w ■ then  discuss  the  rules 
used  for  combining  the  assessments. 

Rules  for  Decomposition  and  Assessment 

When  a problem  is  decomposed  into  its  relevant  compo- 
nents and  structured  in  the  form  of  a probability  diagram, 
it  is  necessary  to  assess  the  conditional  probabilities  for 
each  of  the  branches.  In  order  to  do  this  correctly,  it  is 
first  necessary  to  become  familiar  with  the  following  rules 

Rule  #1  All  branches  from  a particular  node  must 

be  mutually  exclusive. 

This  means  that  a particular  event  cannot  appear  twice, 
as  would  be  true  of  two  different  branches  containing  the 
same  outcome.  Consider  a series  of  probability  diagrams 
which  represent  the  outcomes  from  the  roll  of  a die. 

Figure  4-4  is  incorrect  because  the  branches  are  not  mutually 
exclusive,  but  all  three  probability  diagrams  in  Figure  4-5 
show  mutually  exclusive  branches  and  therefore  are  correct. 
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WHERE  A = (1,2.  3.  4) 
B = (4, 5,  6) 


Figure  4-4 

INCORRECT  PROBABILITY  DIAGRAM  WITH 
BRANCHES  NOT  MUTUALLY  EXCLUSIVE 


WHERE  A = (1,2,3) 
B =■  (4, 5.  6) 


WHERE  A«(1) 

B * (2,  3,  4, 5,  6) 


Figure  4-5 

CORRECT  PROBABILITY  DIAGRAMS  WITH  BRANCHES  MUTUALLY  EXCLUSIVE 


A r.econd  example  represents  a diagram  by  an  analyst  trying 
to  determine  the  speed  of  a new  Russian  aircraft.  The 
branches  in  Figure  4-6  are  not  mutually  exclusive,  and 
therefore  are  incorrect,  while  the  branches  in  Figure  4-7 
are  mutually  exclusive  and  therefore  correct. 

SPEED  >1000  MPH 


Figure  4-6 

INCORRECT  PROBABILITY  DIAGRAM  WITH 
BRANCHES  NOT  MUTUALLY  EXCLUSIVE 


SPEED  >1000  MPH 


Figure  4-7 

CORRECT  PROBABILITY  DIAGRAM  WITH 
BRANCHES  MUTUALLY  EXCLUSIVE 
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Rule  #2  All  branches  stemning  from  a particular 

node  must  be  exhaustive. 

This  ' cans  that  every  possible  outcome  must  be  included. 
In  the  case  of  the  die  problem.  Figure  4-8  would  be  an  in- 
correct representation  because  it  does  not  include  the  case 
of  rolling  a 2. 


ROLL  a 1 


F igure  4-8 

INCORRECT  PROBABILITY  DIAGRAM 
WITH  BRANCHES  NOT  EXHAUSTIVE 


Another  example  depicts  the  determination  of  the  likelihood 
of  the  Soviet  Union's  targeting  an  ICBM  at  the  U.S.  or  at 
Europe,  Figure  4-§  is  not  exhaustive  because  it  does  not 
consider  the  possibility  of  the  missiles  also  being  aimed  at 
another  region  of  the  world,  as  portrayed  by  the  dotted  line. 


AIMED  AT  U.S. 


It 


Figure  4-9 

INCORRECT  PROBABILITY  DIAGRAM 
WITH  BRANCHES  NOT  EXHAUSTIVE 
(CORRECT  WHEN  DOTTED  LINE  IS  ADDED) 
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Rule  #3 


The  probabilities  branching  from  a 
particular  node  must  sum  to  one. 


This  rule  may  serve  as  a check  to  verify  whether  the 
branches  on  the  probability  diagram  are  truly  mutually 
exclusive  and  exhaustive.  It  also  serves  as  a partial  check 
on  the  consistency  of  the  assessor's  judgments,  for  if  the 
diagram  is  correct,  it  is  inconsistent  to  assign  probabilities 
that  do  not  sum  to  one. 

Conditional  Probabilities 

consider  Figure  4-10,  which  refers  to  the  original 
question  concerning  the  likelihood  of  rain.  When  asked 
for  estimates  concerning  the  probability  of  rain  from  a 
global  viewpoint,  our  weatherman  was  unsure,  but  estimated 


RAIN 


Figure  4*10 
SIMPLE  DIAGRAM 


a 60%  chance  of  rain.  His  estimate  of  * .60  is  an  un- 
conditional probability,  so  called  because  it  is  not  based 
on  any  previous  assumptions,  or  branches,  in  the  probability 
diagram. 

Next,  the  weatherman  considered  the  upper  path  of  Figure 
4-3.  First,  he  assessed  the  probability  of  low  pressure  as 
0.8;  this  is  an  unconditional  probability.  Then,  he  assessed 
the  probability  of  NW  winds  after  assuming  that  a low-pressure 
area  occurs  for  sure.  This  is  a conditional  probability,  which 
can  be  written; 

p(NW  winds  I low  pressure), 

the  vertical  line  indicating  "given,"  The  expression  is  read 
"the  probability  of  NW  winds  given  the  occurrence  of  a low- 
pressure  area."  Turning  to  the  upper  right  branch,  the 
weatherman  asked,  "What  is  the  probability  of  rain  assuming 
NW  winds  and  low  pressure  occur  for  certain?"  This,  too,  is 
a conditional  probability,  and  he  assigned  a value  of  0.9: 
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p(Rain|low  pressure  and  NW  winds)  «=  0.9 

Probabilities  for  the  three  events  on  the  upper  path  are 
shown  in  Figure  4-11. 

Strictly  speaking/  that  first  unconditional  probability 
of  0,8  for  lew  pressure  is  really  a conditional  probability; 
it  is  conditional  on  the  person  making  the  assessment # on 
that  person's  expertise/  and  on  the  information  available  to 
the  assessor,  like  today's  weather.  In  this  sense,  all  proba- 
bilities are  conditional.  But  to  simplify  communication,  we 
rarely  describe  all  the  conditioning  events,  and  when  we  do 
not  explicitly  describe  any  of  them,  we  say  that  the  proba- 
bility is  "unconditional.” 


RAIN 


\ 

\ 


Figure  4-11 

UPPER  PATH  OF  FIGURE  4-3 
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In  words,  here  Is  the  description  of  each  probability  shown  in 
Figure  4-11: 

P,  - Unconditional  probability  of  a low-pressure  area 
occurring  tomorrow; 

P2  - Conditional  probability  of  NW  winds  on  the  assump- 
tion there  will  definitely  be  a low-pressure  area, 
or  p(NW  winds  I low  pressure); 

P^  - Conditional  probability  of  SW  winds  on  the  assump- 
tion there  will  definitely  be  a low-pressure  area, 
or  p(SW  winds  I low  pressure); 

P.  - Conditional  probability  of  rain  given  low  pressure 
and  NW  winds,  or  p (Rain | low  pressure  and  NVf  winds); 

Pc  - Conditional  probability  of  no  rain  given  low  pressure 
and  NW  winds,  or  p(No  rain | low  pressure  and  NW 
winds) ; 

Fg  - Conditional  probability  of  rain  given  low  pressure 
and  SW  winds,  or  p (Rain [low  pressure  and  SW  winds); 

P-  - Conditional  probability  of  no  rain  given  low  pressure 
and  SW  winds,  or  p(No  rain | low  pressure  and  SW  winds). 

Next,  the  weatherman  completes  assessments  for  all  the 
remaining  branches.  His  probabilities  are  shown  on  the 
corresponding  branches  in  Figure  4-12. 

The  assessment  procedure  can  be  summarized  as  a fourth 

rule: 

Rule  #4:  In  determining  the  probability  for  an 

event  on  a given  branch  of  the  probability 
diagram,  the  assessor  must  assume  that  all 
events  on  the  path  leading  up  to  the  given 
branch  have  definitely  occurred. 

Independence 

No  discussion  of  conditional  probability  is  complete 
without  considering  the  notion  of  independence.  Since  the 
definition  of  independence  uses  the  idea  of  a conditional 
probability,  we  introduce  the  definition  here  even  though  we 
shall  not  use  it  until  later  in  the  chapter. 

Definition;  Two  events  are  independent  if  the  prob2d3ility 

assigned  to  one  event  is  unaffected  by  knowledge  that  the 

other  event  has  occurred. 
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For  example,  to  the  event  low  pressure,  we  assigned  a proba- 
bility of  0.8.  Presumably,  that  probability  remains  unchanged 
when  the  weatherman  is  told  that  his  favorite  football  team 
has  just  lost  to  their  arch  rivals.  If  that  is  so,  then  the 
event  "low  pressure  tomorrow"  is  independent  of  the  event 
"Team  X has  lost  to  Team  Y today."  Here,  independence  is 
obvious  because  one  event  has  no  causal  relationship  to  the 
other . 


Lack  of  causality  is  not,  however,  an  adequate  basis  for 
judging  independence.  It  is  quite  possible  for  two  events  to 
have  no  causal  connection,  yet  be  judged  non-independent.  For 
example,  without  looking  at  my  wristwatch,  I judge  there  is  a 
0.6  probability  that  the  time  on  my  watch  is  within  five 
minutes  of  3:00  p.m.  If  you  look  at  your  watch  and  tell  me 
it  is  2:58  p.m.,  I now  chsmge  my  probability  to  0.95  (your 
watch  could  be  a little  fast!).  Since  I have  changed  my 
probability  for  the  event  "3:00  p.m.  plus  or  minus  five  minutes 
by  my  watch"  when  I heard  about  another  event,  the  time  shown 
on  your  watch,  we  conclude  that  the  events  are  not  independent. 
Yet,  in  no  sense  can  one  of  our  watches  be  said  to  "cause"  the 
time  shown  on  the  other.  The  times  shown  on  our  watches  are 
correlated,  they  "go  together"  in  an  orderly  fashion,  and  it 
is  this  correlation  that  leads  us  to  judge  the  events  as  non- 
independent. 

In  short,  we  cannot  judge  events  to  be  independent  because 
there  is  no  causal  relationship  between  them.  Instead,  we 
must  ask  if  our  uncertainty  about  an  event  is  changed  by 
knowledge  about  some  other  event.  If  that  extra  knowledge 
does  not  change  our  uncertainty,  then  the  events  are  indepen- 
dent, If  it  does,  then  the  events  are  dependent.  This  is  the 
only  adequate  test  of  independence. 

Rules  for  Combining  Probabilities 

We  return  now  to  the  main  problem  at  hand,  that  of  finding 
I the  unconditional  probability  ot  rain.  To  calculate  this  from" 
the  assessments  shown  on  the  probability  diagrams  of  Figure 
4-12,  we  must  apply  two  important  rules  of  probability. 

Multiplication  Rule;  To  obtain  the  probability  that 

all  events  on  a path  through  the 
probability  diagram  will  occur, 
multiply  together  all  the  individual 
probabilities  on  the  path. 

For  example,  the  probability  for  the  upper  path  is  obtained  by 
multiplying  0.8  by  0.7  by  0.9; 

p(low  pressure  and  NW  winds  and  rain) 

= 0.8  X 0.7  X 0.9 
=.  0.504 


92 


That  is,  there  is  a 50.4%  chance  that  all  three  events  on 
the  path  will  occur.  This  proOability  is  called  a path 
probability.  Repeated  applications  of  the  multiplication 
rule  gives  all  the  path  probabilities  shown  in  the  right 
column  of  Figure  4-12. 

Now  we  are  ready  to  determine  the  probability  of  rain. 

We  need  only  one  more  rule. 

Addition  Rule;  For  mutual ly-exclusive  events  or 

paths,  the  probability  that  one  or 
the  other  will  occur  is  obtained  by 
adding  their  individual  pit'babilities. 

For  example,  what  is  the  probability  that  a fair  die,  when 
rolled,  will  show  an  odd  number?  Since  there  are  six  possible 
outcomes  which  are  mutually  exclusive,  we  can  apply  the  addi- 
tion rule.  The  probability  of  an  odd  number  is  the  same  as  the 
sum  of  the  probabilities  of  a 1,  or  of  a 3,  or  of  a 5 showing. 
The  probability  of  each  of  those  outcomes  is  1/6.  Thus,  by 
the  addition  rule,  the  probability  of  an  odd  number  must  be 
1/6  + 1/6  + 1/6  or  1/2. 

p (odd  number)  =»  p(l  or  3 or  5) 

a p(l)  + p(3)  + p(5) 

= 1/6  + 1/6  + 1/6 

"1/2  PATH  PROS. 


Figure  4-12 

rOMPlPTE  PRORARILITY  DIAGRAM 


Applying  the  addition  rule  to  the  weather  problem,  we 
find  the  probability  of  rain  by  adding  the  path  probabilities 
for  every  path  that  leads  to  rain  (the  paths  are  mutually 
exclusive) : 

P (rain)  = ‘504  + .168  + .048  + .024 
= .74  (or  74%). 

For  no  rain  : 

P (no  rain)  “ .056  + .072  + .032  + .096 
= .26  (or  26%). 

Table  4-1  is  a comparison  of  the  estimates  of  the 
probability  of  rain  made  by  the  weatherman: 


Table  4-1 


liiitially,  the  weatherman  estimated  the  first  probability  of 
rain  as  60%  from  a global  viewpoint  by  considering  all 
factors  "off-the-top-of-the-head. " Finally,  he  assessed  the 
probably  more  accurate  figure  of  74%  by  decomposing  the 
problem  into  its  more  easily  accessible  component  parts. 

Overview 

To  help  tie  together  the  basics  of  probability  diagrams, 
consider  the  following  more  complex  example:  What  is  the 
probability  that  the  Soviet  Union  will  launch  a major  attack 
across  the  border  of  the  People's  Republic  of  China  during 
the  next  twelve  months?  Obviously,  because  of  the  complexity 
of  this  problem,  it  is  very  difficult  to  make  an  off-the-top- 
of-the-head  estimate.  One  possible  method  of  attacking  the 
problem  is  to  break  it  down  into  its  component  parts  and 
derive  separate  probability  assessments  for  each  of  those 
component  parts. 

The  analyst  working  on  this  problem  would  have  to 
decide  which  information  concerning  Sino-Soviet  relations  is 
most  relevant  and  would  have  an  effect  on  the  target  variables. 
Assvune  he  came  to  the  conclusion  that  Russia  will  decide  to 
attack  if  there  is  political  upheaval  in  China,  as  represented 
in  Figure  4-13. 
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RUSSIA  ATTACKS 


The  analyst  might  further  feel  that  the  likelihood  of 
the  occurrence  of  a political  upheaval  will  depend  on  whether 
the  Chinese  political  leader  dies.  The  probability  diagram 
in  Figure  4-14  is  one  possible  representation  of  these 
events . 


RUSSIA  ATTACKS 
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If  this  problem  were  going  to  be  analyzed  in  depth, 
many  more  events  might  be  helpful,  and  all  of  them  would 
have  to  he  very  specifically  defined.  For  our  purpose,  this 
depth,  or  to  the  exact  probabilities,  is  not  necessary;  it  is 
only  the  concepts  with  which  we  are  involved. 

Just  considering  the  top  path  of  the  probability 
diagram,  we  derive  each  of  the  probabilities  as  follows: 

PATH  PROS. 

RUSSIA  ATTACKS  ^ 


- Assessed  probability  of  Chinese  leader  dying. 

P,  - Conditional  probability  of  upheaval  given  the 

leader  dies  - (upheaval | leader  dies). 

P,  - Conditional  probability  of  Russia  attacking 

given  that  the  leader  dies  and  there  is 
upheaval  - (Russia  attacks | leader  dies  and 
upheaval) . 

P4  - Path  probability  representing  the  likelihood 

of  the  whole  branch  occurring  - (Leader  dies 
and  upheaval  and  Russia  attacks)  . 

P4  « Pj^  X P2  X Pj 

= (.20)  X (.40)  X (.80) 
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P.  = .064  (or,  there  is  a 6.4% 

chance  that  all  events  will 
occur  within  the  next  twelve 
months) . 


Again,  after  considering  some  of  the  individual  proba- 
bilities, let  US  go  on  to  the  complete  probability  diagram, 
Figure  4-16. 


RUSSIA  ATTACKS 


PATH  PROB. 


UPHEAVAL 


CHINESE 
LEADER  DIES, 


\ RUSSIA 

DOES  NOT  ATTACK 


.floV  NO  UPHEAVAL 


•80  \ .10, 
\ CHINESE  / 

\leader  LIVEV 


UPHEAVAL 


.90\  NO  UPHEAVAL 


RUSSIA  ATTACKS 


\ RUSSIA 

DOES  NOT  ATTACK 


RUSSIA  ATTACKS 


\ RUSSIA 

.80\  DOES  NOT  ATTACK 

RUSSIA  ATTACKS 


A RUSSIA 

DOES  NOT  ATTACK 


Figure  4-16 

COMPLETE  PROBABILITY  DIAGRAM 


Adding  the  path  probabilities  for  the  four  paths  which 
result  in  a Russian  attack  gives  the  overall  probability  of 
an  attack, 

P(attack)  » .064  + .036  + .016  -f  .072 
» .188 


P(no  attack)  » ,016  + ,084  + .064  + .64B 
- .812 


This  is  the  solution  to  the  original  question,  What  is  the 
probability  that  the  Soviets  will  launch  a major  attack 
across  the  border  of  the  People's  Republic  of  China  during 
the  next  twelve  months?"  Given  our  assumptions  about  the 
relevant  events  and  their  likelihoods  of  occurrence,  the 
overnll  probability  of  attack  is  18. R%  (from  Figure  4-J6). 

This  tlnal  probability  has  been  arrived  at  by  breaking  the 
overall  problem  into  its  small  component  events , events  for 
which  probabilities  can  be  assessed  easily  and  directly. 
Folding  back  these  probabilities  has  made  it  possible  to 
recombine  them  to  determine  their  overall  likelihood  of 
attack,  a likelihood  which  would  have  been  extremely  difficult 
to  obtain  from  an  overall  estimate. 

Folding  Back 

Frequently  there  is  a need  to  know  not  only  the  likeli- 
hood of  any  one  branch  occurring,  but  also  the  likelihood  of 
a aeries  of  branches  occurring  at  one  particular  node.  For 
example,  the  analyst  may  be  interested  in  the  probability 
Russia  will  attack  assuming  the  Chinese  leader  dies.  This 
could  be  determined  by  "folding  back"  to  Node  ^2  the  four 
right-hand  branches  of  Figure  4-16,  the  result  of  which  is 
shown  in  Figure  4-17. 


RUSSIA  ATTACKS 


RUSSIA  ATTACKS 


Figure  4-17 

FOLDING  BACK 


At  Node  #2,  the  new  probability,  P (Russia  attacks | Chinese 
leader  dies),  can  be  calculated  by  summing  the  appropriate 
branches  of  Figure  4-17. 

P (Russia  attacks  I Leader  dies) 

" P (Upheaval j leader  dies)  x 

P (Russia  attacks  I leader  dies  and  upheaval)  + 
P(No  upheaval I leader  dies)  x 
P (Russia  attacks  I leader  dies  and  no  upheaval) 

- (.40)  X (.80)  + (.60)  X (.30) 

- .50 

In  a similar  manner,  the  whole  probability  diagram 
(Figure  4-17)  can  be  folded  back  to  Node  #1  to  detemine  the 
overall  likelihood  that  Russia  will  attack,  P > 19%. 

Sensitivity  Analysis 

We  are  now  ready  to  discuss  the  problem  of  subjective, 
or  personal,  probabilities.  The  reader  will  have  noticed 
that  throughout  the  analysis  of  both  the  weather  and  the 
Russian  problems,  very  little  explanation  was  given  for  the 
assessed  conditional  probabilities.  In  each  case,  the 
weatherman  or  analyst  "assessed"  there  was  a 70%  probability 
of  NW  winds  given  there  would  be  a low-pressure  area,  or 
"assessed"  there  was  a 20%  probability  that  the  Chinese  leader 
would  die  within  twelve  months.  Although  the  analyst  is 
using  his  expertise  to  provide  his  best  assessment,  other 
analysts  may  disagree;  or  the  analyst  may  be  unsure  himself. 

The  question  naturally  arises  about  the  extent  to  which  the 
probability  of  the  target  variable  or  event  is  influenced 
by  any  particular  conditional  probability.  If  the  probability 
of  NW  winds,  given  low-pressure,  is  changed  from  0.7  to,  say, 

0.5  or  0.9,  what  effect  will  this  have  on  the  overall  probability 
of  rain?  If  it  has  only  a small  effect,  then  disagreements 
between  two  experts,  one  of  whom  assigns  0.5  and  the  other  0.9, 
can  be  tolerated.  Or,  if  an  expert  cannot  decide  what  value 
between  0.5  and  0.9  to  assign,  he  can  be  assured  that  it  does 
not  make  any  substantial  difference  so  far  as  the  overall 
probability  of  rain  is  concerned.  Sensitivity  analysis  can 
resolve  these  kinds  of  questions. 

First,  let  us  reconsider  the  weather  problem  and  the 
question  of  whether  the  likelihood  of  NW  winds  given  there 
will  be  a low  pressure  area  has  a substantial  effect  on  the 
overall  probability  of  rain  (see  Figure  4-18). 


Figure  4-18 

PROBABILITY  OF  RAIN 

Possibly  the  weatherman  Is  confident  of  all  probabilities 
except  the  70%.  He  has  enough  Imowledge  or  expertise  to 
feel  comfortable  about  the  other  values > but  not  enough  to 
feel  sure  about  his  assessment  of  70%.  Or  perhaps  he  Is 
discussing  this  analysis  with  two  colleagues  who  are  In 
total  agreement  with  the  analysis  except  for  the  70%  assess- 
ment. How  crucial  Is  70%  with  regard  to  the  overall  proba- 
bility of  rain?  Its  degree  of  sensitivity  can  be  determined 
by  varying  that  probability  over  the  range  of  possible 
values.  If  one  colleague  feels  It  Is  as  low  as  50%  and  the 
other  colleague  thinks  It  is  as  high  as  100%,  the  weatherman 
can  substitute  those  two  extremes  Into  the  diagram  and 
determine  the  overall  effect  on  the  P(RBin).  Table  4-2 
displays  those  results. 


P (NW  Winds  Low  Pressure)  P (Rain) 


1.00  (Max.  value)  .792 

.70  (Original  value)  .744 

.50  (Min.  value)  .712 


Table  4-2 


3 00 


It  is  apparent  from  the  small  change  in  P(Rain),  from 
,71  to  .79,  that  this  overall  probability  is  relatively 
insensitive  to  the  assigned  probability.  Because  of  this 
insensitivity,  the  weatherman  and  his  colleague  do  not  have 
to  concern  themselves  with  refining  the  probability  of  70%. 

If  this  had  been  a sensitive  branch  and  a fluctuation  between 
the  two  extreme  values  did  cause  significant  change  in 
P(Rain),  then  more  discussion  or  information  would  be  needed 
to  ascertain  the  best  value  for  P(NW  winds  Low  pressure) . 

Now  reconsider  the  Russian  problem  where  a 20%  probability 
was  assessed  that  the  Chinese  leader  would  die  within  the 
next  twelve  months  (see  Figure  4-19) . 


RUSSIAN  ATTACK 


If  there  were  a disagreemenr  concerning  this  assessment,  the 
analysts  could  determine  the  possible  range  of  values.  Depending 
on  his  age,  the  minimum  actuarial  probability  of  the  Chinese 
leader's  death  might  be  9%.  The  maximum  might  be  50%,  an 
estimate  supplied  by  an  analyst  who  had  recently  received 
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intelligence  reports  about  the  leader's  deteriorating  health. 
The  effect  of  substituting  these  two  extremes  into  the 
probability  diagram  on  P (Russia  attacking)  is  displayed  in 
Table  4-3. 

P (Leader  dies)  P (Russian  Attack) 


.09 

(Min.  value) 

.15 

.20 

(Original  value) 

.19 

.50 

(Max.  value) 

.30 

Table  4-3 

Unlike  the  outcome  of  the  weather  problem,  the  outcome 
of  this  problem  is  fairly  sensitive  to  changes  in  probability. 
The  probability  of  a Russian  attack  increases  from  15%  to 
30%  as  the  probability  of  the  leader's  dying  increases  from 
its  minimum  to  its  maximum  value.  In  order  to  resolve  this 
disparity  among  the  different  analysts,  it  would  be  necessary 
to  obtain  additional  information  and  to  refine  the  assessment 
of  P (Leader  dies).  This  could  be  done  through  Intelligence 
work  or  possibly  by  a thorough  analysis  by  persons  knowledge- 
able in  this  area.  If  the  target  variable,  a Russian  attack, 
is  sensitive  to  some  of  the  component  probabilities,  then 
the  analysts  must  pay  more  attention  to  those  component 
probabilities  than  to  those  which  are  insensitive. 

Sensitivity  analysis  is  a very  useful  tool  for  helping 
the  analyst  to  determine  which  components  are  crucial  and 
which  are  relatively  insensitive  to  the  analysis.  Refinement 
can  be  done  on  those  decisive  elements  and  less  time  can  be 
spent  on  those  which  are  secondary  in  importance. 

Pruning  Probability  Diagrams 

When  an  analyst  works  with  probability  diagrams  for  any 
length  of  time,  it  becomes  evident  that  this  tool  can  frequently 
evolve  into  a complex  network  of  branches  unsuitable  for 
practical  analysis.  How  often  has  the  analyst  started  with 
a small  set  of  conditioning  events  and  ended  up  with  a 
diagram  containing  16,  32,  or  64  ending  branches,  or  even  a 
diagram  which  contains  over  100,000  path  outcomes?  Since 
the  complexity  of  probability  diagrams  increases  geometri- 
cally with  the  number  of  factors  Considered,  only  10  two- 
valued factors  need  to  be  examined  before  over  1000  paths 
result  (assuming  a symmetrical  diagram) , and  20  such  factors 
will  produce  more  than  1,000,000  ending  branches. 

This  section  is  concerned  with  different  procedures 
that  will  permit  "pruning"  of  probability  diagrams  back  to 
a manageable  size  without  eliminating  pertinent  information. 
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If  the  analyst  uses  these  procedures,  he  will  greatly  increase 
the  applicability  of  probabilitity  diagrams  for  probability 
assessment  and  permit  many  factors  to  be  incorporated,  while 
keeping  the  level  of  analytic  effort  within  reasonable 
bounds. 

Although  the  analyst  probably  will  be  able  to  develop 
his  own  simplification,  the  following  two  basic  pruning 
methodologies  will  be  discussed  in  detail:  adjustment  and 
relaxation  of  assumptions,  and  Markov  chains. 

Adjustment  and  Relaxation  of  Assumptions 

The  central  idea  of  this  method  is  to  direct  the  primary 
thrust  of  the  analysis  to  those  branches  of  the  diagram 
which  possess  very  high  probabilities.  This  amounts  to 
devoting  special  care  to  the  parts  of  the  problem  that 
matter  the  most. 

Consider  first  an  analysis  of  the  likelihood  of  a U.S. 
dollar  devaluation  within  the  next  twelve  months.  Figure  4- 
20  shows  how  the  assessment  of  a possible  devaluation  might 
be  conditioned  on  throe  factors:  inflation,  trade  deficit, 
and  European  pressure. 

To  determine  the  overall  probability  of  devaluation 
from  the  diagram,  it  would  normally  be  necessary  to  assign 
conditional  probabilities  for  each  branch  of  the  tree. 

As  is  evident  from  Figure  4-20,  the  upper  half  of  the  diagram 
is  much  more  important  than  the  lower  half  because  a probability 
of  0.75  has  been  assigned  to  the  event  "inflation ^ 6%/yr. '* 
Therefore,  the  required  probabilities  have  been  assigned  by 
the  analyst  only  to  the  upper  half  of  the  diagram,  for 
reasons  which  will  become  clearer  as  the  analysis  progresses. 
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$ DEVALUED 


Figure  4-20 

PROBABILITY  DIAGRAM  FOR  U.S.  DOLLAR  DEVALUATION 


At  Node  1 in  Figure  4-20,  the  effect  of  the  factors 
subsequent  to  that  node  on  the  two  possible  outcomes  are 
shown  in  Figure  4-21A.  Folding  this  diagram  back  (by 
summing  the  path  probabilities  for  each  outcome)  results  in 
the  simpler,  but  equivalent,  diagram  of  Figure  4-21B. 
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A.  Original  Diagram  Subtaquant  to  Noda  1 


.336 

.084 

.168 

.112 

,090 

.060 

.076 

.076 


B.  Simpiifiad  Diagram  Subtaquant  to  Noda  1 

$ DEVALUED 


FOLDING  BACK  ORIGINAL  DIAGRAM  TO  NODE  1 


The  next  step  wou.ld  normally  be  the  assignment  of 
probabilities  to  each  node  of  the  lower  half  of  the  diagram, 
and  the  folding  back  of  this  half  to  Node  2,  as  was  done 
above  for  the  upper  half  of  the  diagram.  Because  the  two 
halves  of  the  original  diagram  (Figure  4~20)  are  ldent.ical 
in  structure,  (although  the  probabilities  for  each  node 
are  different) , the  diagram  which  results  from  folding  back 


must  resemble  Figure  4-21B.  This  operation  permits  the 
simplification  of  the  original  diagram  to  the  form  shown  in 
Figure  4-22,  and  we  can  approximate  the  probability  of 
devaluation  at  Node  2 (given  inflation  is  leas  than  6%/yr.) 
from  the  known  probability  of  devaluation  at  Node  1 (with 
inflation  equal  to  or  greater  than  6%/yr,). 


$ DEVALUED 


Figure  4-22 

ORIGINAL  DIAGRAM  SIMPLIFIED  BY  FOLDING  BACK 


Assume  the  analyst  feels  the  probability  of  devaluation 
at  Node  2 could  be  as  low  as  25%  or  as  high  as  50%  of  that 
of  Node  1.  The  effect  of  varying  Node  2 from  its  highest 
to  lowest  plausible  values  is  as  follows i 


P (Devaluation) 
Node  1 


0.67 

0.67 


P (Devaluation) 
Node  2 


0.33  (max.) 
0.17  (min.) 


P (Devaluation) 
Overall 


0.5B 

0.54 


It  is  apparent  from  the  small  change  in  the  overall 
probability  of  devaluation  that  it  is  relatively  insensitive 
to  the  assigned  probability  at  Node  2.  Therefore,  for  this 
analysis,  there  is  no  need  for  the  analyst  to  refine  his 
assessment  of  the  probability  at  Node  2 or  to  complete  the 
analysis  by  assessing  individual  probabilities  for  the 
branches  to  the  right  of  that  node. 
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It  should  be  noted  that  the  adjustment  procedure  has 
worked  successfully  on  this  problem  because  of  1)  the  iden- 
tical structure  of  the  two  sections,  and  2)  the  relatively  low 
probability  assigned  to  the  likelihood  of  inflation  being  less 
than  6%  per  year.  If  this  assessment  had  been  much  higher,  it 
would  have  necessitated  assessing  probabilities  for  the  entire 
remaining  portion  of  the  diagram,  and  laboriously  completing 
the  analysis  in  a manner  similar  to  that  employed  for  the 
upper  half  of  the  diagram. 

The  "adjustment"  method  represents  one  instance  of 
relying  on  a precise  analysis  of  highly  probable  branches  of 
a probability  diagram  in  order  to  produce,  as  simply  as 
possible,  a usefully  less  precise  solution  to  the  complete 
diagram.  This  principle  can  also  be  used  in  analyses  where 
there  ip  a sequence  of  highly  probable  assumptions.  The 
effort  is  placed  on  analyzing  tVe  situation  when  the  assump- 
tions hold.  Following  this  analysis,  the  assumptions  are 
"relaxed"  (i.e.,  assumed  not  to  hold)  in  sequence  and  assess- 
ments relevant  to  new  branches  of  the  diagram  are  made. 

Consider  two  cc’ntries  A and  B and  the  possibility  of 
conflict  between  them  resulting  in  open  hostility.  If  there 
are  complex  relations  between  A and  B,  the  analysis  will 
require  a simplification  of  any  probability  diagram  which 
attempts  to  interrelate  major  contributing  factors  leading  to 
possible  hostilities.  In  this  anii-lyais,  we  shall  develop  a 
probability  diagram  based  upon  several  major  assumptions  ard 
then  relax  those  assumptions  one  at  a time.  Each  of  the 
assumptions  to  be  relaxed  must  be  a highly  probable  one. 

In  this  scenario , A is  an  aggressive  but  militarily  weak  I 

country,  while  D is  powerful  but  politically  unstable.  To  s 

understand  how  this  analysis  will  proceed,  consider  the 
following  four  incidents,  which  we  shall  call  events,  and  the 
two  possible  outcomes,  or  hypotheses,  which  can  result  from 
those  incidents: 

Events  ‘ 

1.  Drought  in  B (or  no  drought) . 

2.  Coup  in  D (or  no  coup). 

3.  Arms  shipments  continue  into  A (or  arms  cease). 

4.  Acta  of  terrorism  by  A (or  no  acts  of  ter-  , | 

rorism) . : I 


1 


107 


Hypotheses 


Hostilities  (H) 

No  Hostilities  (H) 

The  analysis  of  this  problem  can  now  proceed  in  two 
directions;  the  analyst  can  work  with  the  four  events  simul- 
taneously by  constructing  a 32-branch  tree  and  assessing  all 
of  the  necessary  probabilities;  or  the  analyst  can  simplify 
the  analysis  by  assuming  each  event  occurs , assessing  the 
probability  of  the  hypotheses  occurring  (given  that  each  event 
has  occurred) , and  then  relaxing  that  assumption  concerning 
the  events  one  at  a time.  This  latter  method  Is  useful  only 
if  some  or  all  of  the  events  are  independent  of  each  other. 
Accordingly/  the  next  step  Is  to  make  judgments  about  event 
Independence . 

Recall  from  the  discussion  of  Independence  earlier  In  this 
chapter  that  two  events  can  be  judged  to  be  Independent  If 
the  probability  assigned  to  one  Is  unaffected  by  knowing  of 
the  occurrence  of  the  other  event.  Looking  down  the  list  of 
events/  the  analyst  decides  that  uncertainty  about  a coup  in 
B would  be  affected  by  knowing  there  Is  a drought.  Here  the 
link  is  causal:  drought  can  lead  to  political  Instability 
which  might  result  In  a coup;  that  Is/  a coup  Is  more  likely 
if  there  Is  a drought  than  if  there  is  no  drought.  In  short/ 
the  events  In  Item  2 depend  on  the  events  in  Item  1. 

Incidentally/  we  should  point  out  to  the  Interested 
reader  that  Independence  Is  a symmetrical  relationship;  If  A 
is  not  Independent  of  h,  then  B Is  not  independent  of  A.  This 
means  that  the  probability  assigned  to  "drought  In  B"  depends 
on  knowing  whether  or  not  there  Is  a coup.  That  may  sound 
odd:  surely  the  occurrence  of  a drought  Is  unaffected  by  the 
occurrence  of  a coup.  True/  as  a statement  of  causality,  but 
as  we  said  earlier  In  the  chapter,  causality  Is  not  an  adequate 
guide  for  judgments  of  independence.  One  must  ask  only 
whether  knowledge  of  one  event  reduces  uncertainty  about  some 
other  evei;t.  In  this  case,  the  assessor  should  ask,  "Is  there 
any  informtion  in  the  event  'coup  in  B'  that  would  affect  my 
uncertainty  about  the  event  'drought  in  B'?"  If  you  had  not 
heard  about  the  weather  and  were  assigning  probabilities  to 
it,  would  you  change  the  probabilities  if  you  were  told  that  a 
coup  occurred?  The  answer  is  yes,  because  droughts  and  coups 
have  a slight  tendency  to  go  together,  though,  of  course, 
not  always.  Knowing  that  either  has  occurred  reduces  our 
uncertainty  about  whether  the  other  has  happened.  We  are 
taking  advantage  of  the  association  between  these  events, 
an  association  which  is  considerably  less  than  perfect, 
yet  useful  for  prediction.  Knowing  a person's  height 
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reduces  your  uncertainty,  but  does  not  completely  eliminate 
it,  about  the  person's  weight.  Height  and  weight  are 
imperfectly  associated:  tall  people  are  heavier,  short 
people  lighter,  but  there  are  exceptions.  Knowing  either 
height  or  weight  reduces  uncertainty  about  the  other  because 
height  and  weight  co-vary.  So  with  droughts  and  coups?  they 
CO- vary.  Thus,  knowing  that  one  has  occurred,  I change  my 
assessment  of  probability  for  the  other. 

Now  we  return  to  the  task  of  identifying  which  events 
in  this  problem  are  independent.  The  analyst  feels  that  the 
events  in  items  3 and  4 are  independent  of  each  other,  that 
acts  of  terrorism,  for  example,  are  just  as  likely  to  occur 
whether  or  not  arms  shipments  continue.  Also,  he  decides 
that  events  in  items  3 and  4 are  independent  of  events  in 
items  1 and  2. 

Having  determined  that  some  events  are  independent,  the 
analyst  recognizes  that  he  does  not  have  to  work  with  the 
full  32-‘branch  tree.  Instead,  he  adopts  the  alternative 
procedure,  which  starts  with  his  assessing  probabilities  for 
the  hypotheses  assuming  that  all  four  events  have  occurred 
(See  Figure  4-23).  Then  he  will  relax  those  assumptions. 


H 


Figure  4-23 

IVIODEL  ASSUMING  EVENTS  1-4  OCCUR 


Assume  the  analyst  has  assessed  the  probability  of 
hostilities,  P(H),  to  be  0.95,  assuming  events  1-4  occur. 
The  next  step  in  the  analysis  is  to  relax  the  assumption 
concerning  event  4 and  construct  the  probability  diagram 
shown  in  Figure  4-24. 


F igure  4-24 

RELAXING  ASSUMPTION  CONCERNING  EVENT  4 


The  analyst  asseaaes  two  new  probabilities,  P (terrorism)  >■ 

0. 90,  and  P(H|no  terrorism)  ■ 0.75,  both  given  that  events 

1,  2,  and  3 have  occurred.  He  then  folds  the  diagram  back 
to  the  simpler  form  shown  In  Figure  4-25. 


The  next  step  In  the  analysis  is  to  relax  the  assump- 
tion concerning  event  3 and  construct  a new  probability 
diagram  (Figure  4-26) . 
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Figure  4-26 

HELAXING  ASSUMPTION  CONCERNING  EVENT  3 


Again,  two  new  probabilities,  P(ani\a  shipment)  =>  0.85,  and 
P(Hlno  arms  shipment)  « 0.60,  both  given  that  events  1 and  2 
have  occurred,  are  assessed  by  the  analyst.  The  resulting 
diagram  can  now  be  folded  back  to  the  diagram  shown  in 
Figure  4-27. 


Figure  4-27 

FOLDING  BACK  RELAXED  ASSUMPTION  FOR  EVENT  3 


At  this  point  in  the  analysis,  with  events  1 and  2 
still  to  be  analyzed,  the  analysis  needs  to  proceed  with 
care.  Up  until  this  point  the  analysis  has  only  been 
concerned  with  events  3 and  4,  which  are  independent.  Now, 
it  must  deal  with  events  1 and  2,  which  are  not  independent. 
If  both  dependant  events  are  regarded  as  very  highly  likely 
to  occur,  the  present  method  may  be  continued,  Otherwise, 
the  diagram  for  the  dependent  events  must  be  constructed  as 
shown  in  Figure  4-28. 


Ill 


COUP 


.81 


Figure  4-28 

DIAGRAM  FOR  ANALYSIS  OF  DEPENDENT  EVENTS  1 AND  2 


Figure  4-28  now  represents  a standard  probability 
diagram  for  which  the  standard  method  of  analysis  is  possible. 
Although  it  still  requires  some  work  for  completion,  that 
work  can  be  accomplished  in  less  time  with  only  an  eight- 
branch  diagram  than  with  the  32-branch  diagram  which  would 
originally  have  been  generated. 

Markov  Chains 

The  second  methodology  for  pruning  probability  diagrams, 
Markov  Chains,  is  an  extremely  uceful  technique  when  the 
analysis  involves  a series  of  events  which  are  repeated  over 
[ a period  of  time.  To  illustrate  its  use,  consider  an  analysis 

[ ) of  Country  C and  its  internal  political  unrest.  The  hypo- 

thesis in  question  is  the  possibility  of  serious  hostilities 
developing  within  the  next  six  months.  If  those  hostilities 
are  to  develop,  they  can  be  considered  to  result  from  oie  of 
the  following  incidents: 
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1. 


Conflicts  between  the  "Inpower"  and  the  "Outpower" 
groups; 


2.  Internal  conflict  among  factions  of  the  "Inpower" 
group; 

3.  Other  (all-inclusive  factor). 

First,  consider  the  probability  diagram  in  Figure  4-29, 
which  traces  out  over  a one-month  period  the  sequence  of 
events  that  can  escalate  the  country  from  its  normal  state 
at  the  beginning  of  the  one-month  period  into  any  one  of  the 
following  three  states  at  the  end  of  that  period:  normal 
state  (N) , tension  state  (T) , and  hostilities  (H) . A one- 
month  period  was  used  because  it  was  felt  that  the  effect  of 
the  incident  would  be  fully  felt  in  this  time  frame. 


IN  VS.  OUT 


^08  T 


INCIDENT 


r.30  INTRA  IN  POWER  XiO  T 


OTHER 


rigure  4-29 

STARTING  IN  NORMAL  STATE  (N) 


Now  consider  a one-month  period  which  begins  with 
Country  C in  a state  of  tension.  The  probability  diagram 
for  this  situation  is  shown  in  Figure  4-30.  Here,  as  before, 
the  country  may  be  in  any  one  of  tha  three  states  (N,  T, 
or  H)  at  the  end  of  the  one-month  period.  Note  that  none  of 
the  probabilities  in  either  Figure  4-29  or  Figure  4-30  are 
conditional  on  events  occurring  prior  to  the  assumed  be- 
ginning states.  This  quality  is  essential  to  the  application 
of  this  method  of  analysis. 
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The  reader  should  note  that  although  many  different  1 
events  can  occur  during  the  one-month  interval,  only  three  1 
possible  end  states  exist:  normal,  tension,  and  hostility.  < 
Because  of  this  limited  range,  the  large  probability  dia-  ] 
goams  (rigures  4-29  and  4-30),  which  are  useful  for  analysis  j 
of  the  many  possible  implications  o£  an  incident,  can  be  1 
pruned,  or  simplified,  by  using  the  diagrams  of  Figure  4-31.  1 
Both  display  the  expected  probabilities  of  terminating  in  ? 
each  of  the  three  possible  end  states.  j 


Figures  4-31.A  and  4-31B  are  representations  of  one- 
month  periods  only.  Since  the  hypothesis  is  concerned  with 
the  probability  of  hostilities  over  a eix-month  period,  we 
must  connect  these  diagrams  together  (link  them  into  a "chain") 
so  that  if,  at  the  end  of  one  month,  the  H branch  has  not 
been  reached,  the  process  starts  over  at  the  beginning  of 
the  next  one-month  period.  Figure  4-32,  which  assumes  that 
Country  C is  in  a normal  state  to  start  with  and  extends  the 
analysis  for  a second  one-morith  period,  combines  these  two 
diagrams. 


Although  considerable  pruning  has  occurred  between 
Figures  4-29  and  4-30,  and  Figure  4-31,  it  is  evident  from 
Figure  4-32  that  as  new  diagrams  are  added  at  the  end  of 
each  one-month  period,  the  number  of  end  points  will  rapidly 
evolve  into  a complex  network  unsuitable  for  practical 
analysis.  At  the  end  of  the  first  month  there  are  3 endpoints, 
7 at  the  end  of  two  months,  and  123  endpoints  at  the  end  of 
six  months. 

Instead  of  drawing  all  123  branches,  the  process  can  be 
simplified  through  the  use  of  a Markov  model,  which  chains 
together  each  of  the  one-month  periods  necessary  to  obtain 
six  months.  This  simplification  can  be  achieved  most  readily 
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by  constructing  a transition  probability  matrix,  such  as 
that  shown  in  Figure  4-33,  which  gives  the  probability  of 
going  from  one  state  to  another  in  a one-month  period. 
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Figure  4-33 

TRANSITION  MATRIX  FOR  ONE-MONTH  PERIOD 


For  example,  assume  you  are  In  state  N:  the  probability  of 
still  being  in  N at  the  end  of  one  month  is  92%;  the  probability 
of  going  into  a state  of  tension  is  5%;  and  the  probability 
of  going  into  hostilities  during  the  one-month  period  is  .03 
or  3%.  Note  that  while  it  is  possible  to  enter  state  H from 
either  N or  T,  once  you  have  entered  H,  according  to  the 
purpose  of  the  analysis,  it  is  impossible  to  leave. 

Implicit  in  this  matrix  is  an  important  underlying 
assumption;  namely,  that  the  probability  of  moving  from  one 
state  to  another  is  dependent  only  on  the  state  at  the 
beginning  of  any  period,  not  oh  liow  that  beginning  state  was 
reached.  Once  the  transition  probabilities  have  been  assessed, 
the  matrix  can  be  used  to  calculate  the  likelihood  of  hostilities 
having  occurred  at  the  end  of  any  number  of  months,  n.  This 
is  done  by  raising  the  matrix  to  the  nth  power,  which  will 
give  you  not  only  the  probability  of  hostilities,  but  all 
the  other  probabilities  associated  with  the  problem  for  the 
nth  month,  in  the  form  of  a new  matrix.  Doing  this  requires 
a knowledge  of  matrix  algebra  and  either  a willingness  to 
perform  the  laborious  computations  manually  or  a computer 
conveniently  programmed  to  do  it  for  you. 
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Figure  4-34  gives  the  results  of  such  a computation  for 
Country  C,  assuming  that  the  starting  state  was  normal  (N) . 
The  probability  of  hostilities  within  the  next  six  months, 
as  read  from  the  graph,  is  17%. 


Figure  4-34 

PROBABILITY  OF  HOSTILITIES  AFTER  SUCCESSIVE  ONE-MONTH  PERIODS 


Hierarchical  Inference 


Up  to  this  point,  we  have  assumed  that  observed  or 
observable  data  can  be  related  directly  to  a given  hypothesis, 
and  that  its  impact  on  the  probability  of  the  hypothesis 
occurring  can  easily  be  determined  by  the  application  of 
Bayes*  Theorem,  However,  the  complexity  of  many  real-world 
inference  problems  may  require  an  amount  or  kind  of  knowledge 
necessary  to  assess  probabilities  or  likelihood  ratios 
directly  linking  all  elements  of  data  to  the  hypothesis  beyond 
the  capability  of  any  one  individual.  In  such  cases,  one 
approach  is  to  decompose  the  complex  problem  by  using  a 
hierarchical  structure.  It  may  be  easier  to  assess  first 
those  probabilities  linking  the  data  to  intermediate  variables 
(activities  and/or  indicators  of  activity}  relevant  to  the 
hypothesis  and  then  assess  the  probabilities  linking  these 
intermediate  variables  to  the  hypotheses. 
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One  of  the  most  compelling  practical  reasons  for  a 
hierarchical  decomposition  is  that  different  individuals  are 
experts  in  different  areas;  organizations  are  hierarchically 
structured  to  take  advantage  of  this  fact.  For  many  Inference 
problems,  it  is  unlikely  that  any  one  Individual  has  the 
necessary  experience  to  relate  the  lower-level  data  or 
observables  to  the  upper- level  hypotheses.  A problem  decom- 
posed along  hierarchical  lines  makes  it  possible  for  the 
knowledge  of  each  expert  to  be  applied  to  the  problem  in  a 
logically  consistent  manner. 

To  a great  extent,  the  structure  of  a hierarchy  reflects 
the  conceptual  model  which  the  inference  maker  has  of  a 
particular  problem.  For  this  reason,  any  particular  structure 
contains  a degree  of  subjectiveness:  the  way  a particular 
individual  chooses  to  decompose  a particular  problem,  and  the 
way  environmental  constraints  may  make  one  choice  more 
appealing  than  another. 

Establishing  a Hierarchical  Structure 


To  illustrate  how  a hierarchical  structure  might  be 
constructed,  let  us  consider  a substantive  example.  Suppose 
that  an  intelligence  analyst  is  evaluating  the  Intent  of 
Country  A to  develop  an  independent  nuclear  weapons ’production 
capability  within  the  next  five  year/i.  This  allows  him  to 
establish  two  hypotheses: 

H.  - Country  A intends  to  develop  a nuclear  weapons 
production  capability  within  5 years,  and 

H2  - Country  A does  not  intend  to  develop  a nuclear 
weapon  production  capability. 

He  then  assembles  a list  of  activities  that  would 
support  these  hypotheses,  such  as  nuclear  research  and  high- 
explosive  research.  The  next  step  is  to  identify  indicators 
related  to  these  activities.  One  such  indicator  of  increased 
nuclear  research  would  be  the  construction  or  expansion  of 
a centrifugal  enrichment  plant  to  increase  the  production 
capability  of  fissionable  material.  The  last  step  in  establishing 
the  structure  is  to  find  data  that  would  confirm  or  deny  the 
activities  or  their  related  Indicators.  An  example  is  a report 
from  a photo  interpreter  describing  the  construction  of  several 
new  cooling  towers  at  the  enrichment  plant.  This  datum,  which 
the  analyst  received  from  an  expert  in  this  technical  area, 
and  which  increases  the  chances  that  the  output  of  enriched 
uranium  will  increase  over  the  next  two  years,  lends  support 
to  Hi  . 


The  analyst  consults  with  experts  in  other  areas  which 
he  considers  relevant  to  his  problem  and,  from  the  information 
obtained,  constructs  the  hierarchical  structure  in  Figure  4- 
35. 

Quantitative  Linkages  Between  Adjacent  Levels 

In  order  to  use  the  observable  Information  and  to  find 
the  probabilities  associated  with  the  two  hypotheses,  it 
is  necessary  to  assess  quantitative  linkages  between  levels. 
The  linkages  will  take  the  form  of  conditional  probabilities 
assessed  by  the  analyst.  The  conditional  probabilities 
refer  to  events  at  the  level  of  interest  given  the  occurrence 
of  an  event  or  the  truth  of  a hypothesis  at  the  next  higher 
level. 


The  assessed  conditional  probabilities  and  likelihood 
ratios  are  measures  of  the  strength  of  relations  eunong 
variables.  If,  for  example,  the  probability  of  an  observed 
datum  is  about  the  same  in  the  presence  as  in  the  absence  of 
an  indicator,  then  it  provides  very  little  Information  about 
whether  or  not  the  indicator  is  in  fact  present.  If,  on  the 
other  hand,  the  two  probabilities  are  quite  different,  then 
the  datum  provides  considerable  information  about  the  presence 
or  absence  of  the  indicator. 

The  assessment  of  quantitative  .linkages  between  levels 
begins  at  the  top  of  the  structure  with  the  hypothesis  to 
activity  or  data  linkages  and  continues  downward  until  a 
linkage  is  established  among  all  of  the  elements  in  each 
level  and  the  elements  in  the  next  higher  level  to  which 
they  directly  relate.  Accordingly,  the  first  step  is  to 
assess  conditional  probabilities  of  data  and  activities  given 
Hj.  and  H-,  for  the  row  of  boxes  just  below  the  top  box,  as 
shown  in'^Figure  4-35. 

For  example,  consider  the  activity,  high-explosive 
Rj^iD  program.  To  assess  the  conditional  probabilities  asso- 
ciated with  this  activity  given  the  truth  of  Hi  or  H2,  the 
analyst  begins  by  assuming  that  Hi  is  true,  that  Country  A 
intends  to  develop  a nuclear  weapons  capability  within  five 
years.  Under  these  circumstances,  what  is  the  likelihood 
that  a high-explosive  R&D  program  would  be  conducted?  The 
analyst  believes  the  chances  to  be  high,  about  95%.  This 
leaves  a 5%  chance  that  such  a progreun  would  not  be  carried 
out.  Next,  the  analyst  assumes  H2  to  be  true,  that  Country  A 
does  not  intend  to  develop  a nuclear  weapons  capability  within 
five  years.  In  this  case,  he  feels  there  is  only  a 35% 
chance  that  the  high-explosive  R&D  program  would  occur,  and, 
therefore,  a 65%  chance  that  it  would  not  occur.  These  assess- 
ments are  shown  in  Figure  4-36. 


H*  ~ COUNTRY  A INTENDS  TO  DEVELOP  A NUCLEAR 
WEAPONS  CAPABILITY  WITHIN  5 YEARS 

Ho  <-  COUNTRY  A DOES  NOT  INTEND  TO  DEVELOP  A 
NUCLEAR  WEAPONS  CAPABILITY  WITHIN  5 YEARS 


DATUM  1 

COUNTRY  A THREATENS 
COUNTRY  B WITH  USE 
OF  “DRASTIC"  WEAPONS 
IF  TERRORISM  CONTINUES 


INDICATOR  1 

ENRICHMENT 

PLANT 

EXPANSION 


PROGRAM 


INDICATOR  2 

INCREASED  USE 
OF  NUCLEAR 
MATERIAL 


DATUM  2 

RAD  PROGRAM 
DIRECTORS  MEET 
IN  WEEK-LONG 
CONFERENCE 


DATUM  3 

NO  OBSERVED 
CHANGE  IN 
NUCLEAR  TEST 
ACTIVITY 


ACTIVITY  2 

HIGH-EXPLOSIVE 
RAO  PROGRAM 


INDICATOR  3 

INCREASED 

SCIENTIFIC 

ACTIVITY 


DATUM  4 

PHOTO-RECON. 
ADDITIONAL 
COOLING  TOWERS 


DATUM  5 

20% INCREASE 
IN  USE  OF 
HEAVY  WATER 


DATUM  6 
SALES  OF 

PLUTONIUM  WATER 
TO  OTHER  COUNTRIES 
CANCELLED 


Figure  4-3B 

HIERARCHICAL  STRUCTURE  FOR 
NUCLEAR  WEAPONS  DEVELOPMENT  PROGRAM 


DATUM  7 

DECREASE  IN 
PUBLICATION  ON 
HIGH-EXPLOSIVE 
RESEARCH 
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Nudeir  No  Nucitir 
Capibility  Capability 


0.96 

0.36 

0.06 

0.66 

Flgura  4-36 

PROBABILITY  MATRIX  LINKING 
ACTIVITY  TO  THE  HYPOTHESES 


In  order  to  carry  the  analysis  through  to  solution,  similar 
assessments  relating  the  second  activity  and  the  first  two 
data  to  the  hypotheses  are  made  by  the  analyst. 

The  next  stop  Is  to  assess  the  conditional  probabilities 
linking  data  aj\d  indicators  in  the  third  level  to  activities 
In  the  second  level.  For  example,  the  analyst  feels  almost 
sure  that  ^ the  nuclear  R&D  program  were  being  conducted, 
then  there  would  be  an  increase  in  the  use  of  nuclear  material. 
He  assigns  a likelihood  of  0.80.  On  the  other  hand,  he 
feels  that  there  Is  a 60%  chance  that  Increased  use  would  occur 
even  without  the  RiD  program.  Figure  4-37  shows  the  matrix 
representing  these  assessments. 


IncrMwd  Uu 
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No 
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R&D  R&D 


O.RO 

0.60 

0.20 

0.40 

Figure  4-37 

PROBABILITY  MATRIX  LINKING 
INDICATOR  2 TO  ACTIVITY  1 

\ 
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Again,  similar  assessments  were  made  for  indicators  1 and 
3 and  for  datum  3. 

The  third  step  is  the  assessment  of  conditional  proba- 
bilities linking  data  in  the  fourth  level  to  indicators  in 
the  third  level.  As  before,  there  will  be  a number  of 
assessments  required  to  link  all  the  data  to  the  hierarchical 
structure.  Figure  4-38  shows  the  probabilities  linking  a 20% 
increase  in  the  use  of  heavy  water,  to  the  inaicator  of  in- 
creased use  of  nuclear  material. 


20K  IncrtMtin 
Uia  of  Huvy  Watar 


incraaia  in 
Um  of 

Nuelaar  No 
Malarial  Incraati 


0.99 


0.50 


Figure  4-38 

CONDITIONAL  PROBABILITIES  LINKING 
DATUM  e TO  INDICATOR  2 


The  analyst  believes  there  is  a 99%  chance  that  Increased 
use  of  heavy  water  will  be.  observed  if  there  is  a general 
increase  in  the  use  of  nuclear  material  and  only  a 50% 
chance  of  observing  this  datum  otherwise. 

Although  it  is  possible,  as  indicated  in  Figure  4-38, 
to  assess  conditional  probabilities  linking  data  to  the 
appropriate  level  of  the  hierarchical  structure,  it  is 
typically  the  case  that  any  given  datum  is  very  unlikely 
given  any  hypothesized  conditioning  event.  Analysts  therefore 
often  find  it  easier  to  assess  a ratio  of  conditional  proba- 
bilities for  a datum  than  to  assejs  bKe  conditional  probabilities 
themselves. 

For  example,  if  the  analyst  wishes  to  assess  such  a 
ratio  for  the  link  between  datura  5 and  indicator  2,  conditional 
probabilities  of  which  are  given  in  Figure  4-30,  the  first 
step  is  to  ask,  "Would  a 20%  increase  in  the  use  of  heavy 
water  be  more  likely  to  occur  if  there  was  a general  increase 
in  the  use  of  nuclear  material  or  if  there  were  no  such  general 
increase?"  Clearly,  that  datum  is  the  more  likely  result  of 
a general  increase  in  the  use  of  nuclear  material.  Next,  the 
analyst  asks,  "How  much  more  likely?"  In  this  case,  he  might 
say  he  thought  the  20%  increase  in  heavy  water  usage  to  be 
about  twice  as  likely  for  a general  increase  as  for  no  increase. 
This  likelihood  ratio  is  shown  in  Figure  4-39.  Strictly 
speaking,  because  the  likelihoods  shown  in  Figure  4-38,  0.99 
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and  0.50,  are  in  a ratio  of  0.99/0.50  « 1.98,  and  the  assessed 
likelihood  ratio  shown  in  Figure  4-39  is  2.0,  the  analyst 
has  been  inconsistent  in  his  judgments.  However,  the  incon- 
sistency is  very  slight  and  is  easily  attributable  to  judg- 
mental error  that  often  arises  when  assessors  round  off 
probability  judgments  to  the  nearest  0.05,  or  ratios  to  the 
nearest  unit. 

Either  likelihoods  or  likelihood  ratios  can  be  assessed 
in  the  inference  problem  since  what  is  important  is  the 
ratio  of  the  numbers  rather  than  their  absolute  value. 
Whichever  type  of  judgment  seems  easiest  or  most  natural  to 
the  assessor  is  the  one  that  should  be  used. 
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Figure  4-39 

NORMALIZED  LIKELIHOOD  RATIO 
SHOWING  THE  RELATIONSHIP  OF 
DATUM  B TO  INDICATOR  2 

There  remains  one  other  probability  assessment  necessary 
to  the  solution  of  the  hierarchical  inference  problem.  This 
is  the  analyst's  assessment  of  the  prior  probabilities  of 
the  hypotheses,  without  regard  for  information  included  in 
the  hierarchical  structure.  He  believes  that  and  H2 
are  equally  likely  prior  to  determining  the  inference  from 
the  available  data.  His  prior  odds  are,  therefore, 

P(Hi)  0.50  1 

P(H2)  “ 0.50  " I * 

The  purpose  of  the  inference  is  to  update  or  modify  these 
prior  probabilities  by  utilizing  the  explicit  evidence 
available  in  the  lower-level  observables. 

Once  the  analyst  completed  the  task  of  assessing  the 
conditional  probabilities  relating  all  the  levels  of  the 
hierarchical  structure,  he  assembled  them  in  the  form  of  a 
deductive  hierarchical  structure,  shown  in  Figure  4-40. 

Mathematical  Solution  of  the  Hierarchical  Inference  Problem 

I 

The  analyst,  having  constructed  the  deductive  structure 
from  the  top  down,  must  now  solve  the  problem  starting  at 
the  bottom  of  the  structure  and  working  up,  to  determine  the 
likelihood  of  all  the  data,  D,  given  hypotheses  Hi  and  H2. 


The  inductive  hierarchical  structure,  represented  in 
Figure  4-41,  shows  the  steps  and  the  computational  results 
which  the  analyst  went  through  to  obtain  a solution.  (The 
principles  behind  those  computations  are  given  in  Appendix  D, 
along  with  a case  study  in  hierarchical  inference.)  For  each 
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Figure  4-41 

INDUCTIVE  HIERARCHICAL  STRUCTURE  FOR 
NUCLEAR  WEAPONS  DEVEl.OPMENT  PROGRAM 


box,  the  two  numbers  show  the  relative  likelihood  of  all  the 
data  D below  that  point  in  the  structure,  given  that  one  of 
the  next  level  events  occurs.  The  top  row  vector  shows  that 
the  final  likelihood  ratio  was  found  to  be 

P(D|Hi)  4.48  3.1 


P(D1Hj_)  1.45  1 
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since  the  prior  odds  were  assessed  as  1:1,  the  posterior 
odds  are  obtained  using  Bayes'  Theorem: 


P(Hi|D)  P(D|Hj^)  P(Hj^)  J.l  1 

PTiJI^  " P(D|H2)  P(H2)  “ ” * I 


This  indicates  that  Is  now  more  than  three  times  as 
likely  as  H2«  or  expressed  as  probabilities - 

P(H3^|D)  - 3.1  “ 0.76,  and 

TTl 


P(H2|d)  - 1 « 0.24. 

Tn 


Observation  of  those  seven  items  of  data  leads  the 
analyst  to  believe  there  is  a 76%  chance  that  Country  A intends 
to  develop  a nuclear  weapons  capability  within  five  years. 

That  inference  is  the  logical  result  of  his  assessments  for 
the  pieces  of  the  decomposed  problem. 
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CHAPTER  5 


THE  VALUE  OF  INFORMATION 


Decisions  on  whether  or  not  to  gather  new  information, 
and  if  so,  how  much,  can  be  analyzed  in  exactly  the  same  way 
as  any  other  decision  by  laying  out  a decision  diagram  with 
appropriate  acta,  events,  probabilities,  and  outcome  values, 
and  then  analyzing  its  implications,  along  the  lines  sketched 
in  Chapters  1 and  2 of  this  Handbook.  However,  the  type  of 
linking  which  goes  into  such  an  analysis,  and  the  kinds  of 
problems  which  often  arise,  merit  some  special  considera- 
tion. 


Information  decisions  differ  from  what  we  might  call 
primary  decisions  in  that  primary  decisions  lead  directly  to 
some  outcome,  whereas  information  decisions  lead  directi''  to 
the  gathering  of  more  data  prior  to  making  a subsequent 
decision.  The  assumption  is  that  additional  information 
will  increase  the  chances  of  a desirable  outcome.  If  the 
resulting  expected  improvement  in  the  value  of  primary 
decisions  exceeds  the  expected  cost  of  the  information 
gathering,  then  that  difference  is  a measure  of  the  net 
value  of  the  information. 

To  illustrate  how  a specific  analysis  of  the  value  of 
information  might  proceed,  we  shall  take  a quite  realistic 
example  from  the  foreign  policy /defense  field. 

Sui pose  that  in  the  course  of  a tense  situation  in 
Cyprus  liable  to  lead  to  overt  hostilities,  the  Commander- 
in-Chief  tor  Europe  (CINCEUR)  has  received  a directive  from 
the  Joint  Chiefs  of  Staff  (JCS)  instructing  him  to  prepare 
to  evacuate  U.S.  civilians  from  Cyprus  only,  or  from  Cyprus, 
Greece  and  Turkey,  on  three  hours'  notice.  The  evacuation 
notice,  if  it  comes,  will  be  a further  JCS  directive,  pre- 
sumably based  on  international  developments  focused  on 
Cyprus  and  related  to  the  possibility  of  conflict  between 
Greek  and  Turkish  Cypriots.  There  are  three  evacuation 
directive  possibilities;  major  (Em) , involving  evacuation 
from  all  three  countries,  Cyprus  only  (Ec) , and  none  (En)  . 

The  primary  decision  facing  CINCEUR  is  what  posture  of 
readiness  to  adopt  in  the  anticipation  of  such  an  evacuation 
directive.  The  alternatives  can  be  characterized  by  three 
basic  postures;  high,  medium,  and  low,  symbolized  as  Ph, 

Pl*  ''^'he  kinds  of  action  each  posture  would  entail  are 
shown  on  the  left  of  Figure  5-1,  and  an  evaluation  of  some 
cf  their  costs  and  other  implications  are  shown  on  the 
right. 
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Figure  5-1 

PRIMARY  DECISION  FACTORS 


Tho  value  of  these  posture  decisions  is  strongly  influ-  j 

enced  by  whether  there  wil]  be  an  evacuation  directive  and,  j 

xf  so,  whether  it  is  major  or  involves  Cyprus  only.  In  1 

genf»ral,  the  higher  the  posture,  the  greater  the  cost  (in  | 

money  and  non-monetary  considerations)  , but  also  the  gre^'ter 
the  net  benefit  if  a major  evacuation  is  ordered.  ' 

. 

CINCEUR  could  decide  immediately  on  what  readiness  ’ 

posture  to  adopt.  Figure  5-2  shows  a decision  diagram  that  i 

might  reasonably  reflect  his  best  current  judgments.  The 
dollar  values  to  the  right  incorporate  judgmental  adjust- 
ments for  iion-monetary  factors  such  as  U.S.  civilian  exposure.  ,, 

As  we  would  expect,  the  values  become  more  negative  as  the  | 

evacuation  situation  becomes  more  serious.  Leas  obvious  is  | 

**■>  the  change  in  values  as  the  posture  becomes  lower:  although  | 

' the  cost  of  the  posture  is  lower  for  less  readiness,  the  | 

exposure  costs  increase  disproportionately,  with  the  result  1 

that  the  overall  values  are  higher  for  the  lesser  postures.  I 

On  these  judgments,  we  see  that  medium  posture  (E^)  has  ] 

least  expected  cost  ($-13. OM)  and  should  therefore  be  pre-  I 

f erred  if  the  posture  decision  has  to  be  made  now.  J 
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Figure  5-2 

DIAGRAM  OF  PRIMARY  DECISION 


Intelligence  Collection  Decision 

CINCEUR  may  have  the  option  of  gathering  information, 
before  making  the  posture  decision,  by  varying  the  levels  of 
his  intelligence  collection  effort.  He  can  direct  a heavy 
collection  effort  (Ch) , a moderate  effort  (C^) , or  a low 
effort  (Cl).  Clearly,  the  more  intelligence  he  collects, 
the  more  likely  he  is  to  receive  information  diagnostic  of 
the  prospects  for  evacuation  and,  therefore,  the  more  likely 
it  will  be  that  his  uncertainty  about  the  proper  course  of 
action  will  be  reduced.  On  the  other  hand,  intelligence 
collection  is  costly  in  terms  of  money  and  resources  diverted 
from  other  uses.  The  question  is,  at  what  level  of  intelli- 
gence collection  is  the  best  balance  of  cost  and  information 
value  achieved? 
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These  kinds  of  considerations,  briefly  and  informally 
stated  above,  would  be  taken  into  account  by  CINCEUR  in 
making  his  intelligence  collection  decision,  whether  or  not 
he  uses  decision  analysis.  We  shall  now  see  how  these 
considerations  can  be  analyzed  on  a decision  diagram.  The 
general  structure  of  a decision  diagram  corresponding  to  the 
above  problem  is  given  in  schematic  form  in  Figure  5-3.  The 
information  to  be  received  is  classified  as  being:  highly 
indicative  of  impending  hostilities  (1^)  , moderately  indica'- 
tive  diti)  , or  least  indicative  (1l)  • 

In  conformity  with  the  general  rule  for  drawing  deci- 
sion diagrams,  the  order  of  acts  and  events  is  according  to 
when  the  acts  are  taken  and  when  the  events  are  known  to  the 
decision  maker.  Thus,  the  immediate  information  collection 
(C)  act  fork  is  followed  by  the  information  received  (I) 
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Figure  5-3 

SCHEMATIC  DIAGRAM  FOR  INFORMATION  COLLECTION  DECISION 
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event  forks , followed  by  the  readiness  posture  (P)  act 
forks/  followed  by  the  evacuation  situation  (E)  event  forks. 
Each  fork  happens  to  have  three  branches,  corresponding  to 
three  act  options  or  event  possibilities,  but  that  does  not 
have  to  be  the  case  and  generally  is  not.  Any  path  through 
the  tree  corresponds  to  one  possible  sequence  of  acts  and 
events  leading  to  outcomes,  numbered  from  1 to  81,  whose 
values  are  to  be  assigned.  For  example,  the  path  repre- 
sented by  the  upper  bounding  branches  of  the  tree  corre- 
sponds to  a sequence  where  a high  level  of  collection  effort 
has  been  undertaken  (Ch) , the  information  received  turns  out 
to  be  highly  indicative  of  hostilities  (Ih) / a high  readiness 
posture  is  taken  by  CZNCEbR  (Pg) , and  a situation  develops 
which  results  in  a directive  from  JCS  to  undertake  major 
evacuation  (E^) . This  outcome  is  numbered  1,  and  its  value 
will  be  recorded  in  the  last  column. 

uantifying  the  Decision  Diagram 


In  principle,  the  problem  is  now  simply  one  of  assigning 
appropriate  values  and  probabilities  to  the  diagram  and 
folding  it  back  to  find  the  immediate  information  collection 
decision  with  the  highest  expected  value.  The  required 
probabilities  for  the  two  types  of  event  forks  are  shown  in 
the  probability  diagram  in  Figure  5-4A.  The  expert  needs  to 
know  what  the  probabilities  of  receiving  the  three  different 
levels  of  information  are,  regardless  of  the  subsequent 
evacuation  situation.  He  also  needs  to  know  the  proba- 
bilities of  the  various  evacuation  situations  g^ven  the 
particular  level  of  information  received.  In  principle, 
there  is  no  reason  why  CINCEUR  could  not  assess  these  proba- 
bilities directly.  The  decision  diagram  could  then  be 
folded  back  by  using  the  techniques  described  in  Chapter  1 
to  give  the  expected  values  of  the  various  courses  of  action. 

For  this  problem,  however,  it  is  much  easier  for  the 
expert  to  assess  the  probability  of  receiving  the  various 
levels  of  information,  given  that  the  various  evacuation 
situations  occur,  than  it  is  for  him  to  assess  the  proba- 
bilities of  the  evacuation  situations,  given  that  he  has 
received  the  information.  Similarly,  it  is  easier  for  him 
to  assess  the  probability  of  evacuation  situations  regard- 
less of  the  information  received.  In  other  words,  the 
probabilities  which  are  most  natural  for  the  decision 
maker  to  assess  are  just  the  reverse  of  the  probabilities 
required  by  the  decision  diagraun.  This  suggests  drawing  the 
probability  diagram  shown  in  Figure  5-4B  and  then  folding  it 
back  by  using  Bayes'  Theorem  and  the  techniques  outlined  in 
Chapter  4.  This  procedure  will  yield  the  required  proba- 
bilities. 
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A.  Required  Probabilities  B.  Readily  Assessed  Probabilities 


I Figure  5-4 

PROBABILITIES  FOR  INFORMATION  DECISION  DIAGRAM 

1 

I There  la  a further  complication  In  the  present  case, 

however.  The  evacuation  situation,  which  Is  the  state  of 
affairs  about  which  the  collection  system  gathers  Informa- 
tion, is  not  observable  directly.  Instead,  various  levels 
of  activity,  such  as  fleet  activities,  military  leaves, 
radio  broadcasts,  travel  restrictions,  navigation  aids,  and 
the  like,  are  observed  which  provide  some  clue  as  to  the 
true  evacuation  situation.  Thus,  the  Information  system 
imperfectly  reports  about  activity  levels  which,  in  turn, 
aru  probabilistically  related  to  the  true  evacuation  situa- 
tion. This  kind  of  an  inference  structure  calls  for  the  use 
of  hierarchical  inference  (as  discussed  in  Chapter  4)  to 
obtain  the  required  probabilities.  Figure  5-5  shows  the 
hierarchical  structure  of  an  information  system  for  a par- 
ticular level  of  collection  effort.  Using  this  structure, 
the  expert  assesses  first  the  prior  distribution,  that  is, 
the  probability  that  each  of  the  evacuation  situations  will 
occur;  then,  given  a particular  evacuation  situation,  he 
assesses  the  probability  that  each  of  the  three  levels  of 
activity  will  occur;  and  finally,  for  each  activity  level, 
he  assesses  the  probability  that  each  of  the  different  kinds 
of  information  will  be  received.  The  dotted  line  surrounding 
the  activity  level  and  information  received  forks  indicates 
that  the  probability  of  various  levels  of  information  being 
I received,,  given  the  activity  levels,  is  a function  of  the 

particular  level  of  collection  employed.  The  other  proba- 
bilistic connection  between  the  evacuation  situation  and  the 
activity  level  is  not  under  the  control  of  the  information 
system.  Because  there  are  three  levels  of  collection  being 
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LEVEL  OF  COLLECTION  C 


Figure  5-5 

HIERARCHICAL  STRUCTURE  OF  INFORMATION  SYSTEM 


considered  in  the  present  case,  there  will  be  throe  proba- 
bility diagrams  which  have  identical  probabilities  relating 
the  activity  levels  to  the  evacuation  situations,  but  which 
have  different  probabilities  relating  the  observed  informa- 
tion to  the  activity  levels,  which  are  dependent  on  the 
capability  of  the  particular  level  of  collection. 

Figure  5-6  displays  the  assessed  probabilities  in 
matrix  form  for  the  present  case.  The  expert  has  assessed 
prior  probabilities  of  5%  that  a major  evacuation  situation 
will  occur,  55%  that  a "Cyprus  only"  evacuation  situation 
will  occur,  and  40%  that  no  evacuation  will  become 
necessary.  The  first  column  of  the  top  matrix  shows  that, 
given  the  occurrence  of  a major  evacuation  situation,  there 
is  an  80%  chance  of  a high  level  of  activity,  an  18%  chance 
of  a medium  level  of  activity,  and  a 2%  chance  of  a low 
level  of  activity.  Similarly,  the  second  column  contains 
the  probability  of  each  of  the  activity  levels  given  that  a 
"Cyprus  only"  evacuation  situation  is  occurring,  and  the 
final  column  contains  the  probabilities  if  the  true  situa- 
tion is  one  in  which  evacuation  is  not  required.  The  three 
matrices  at  the  bottom  reflect  the  extent  to  which  each  of 
the  levels  of  collection  reports  accurately  about  the 
activity  levels.  For  example,  the  high  collection  level, 
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shown  on  the  left*  results  in  a 90%  chance  of  reporting  a 
high  level  of  activity  when  in  fact  there  is  a high  level  of 
activity.  In  contrast*  a medium  collection  effort  has  an 
86%  chance  of  reporting  high  activity  when  it  occurs*  andi 
the  status  quo*  or  low  collection  effort*  has  an  81%  chanoe. 

Figure  5-7  shows  how  the  assessed  probabilities  can  be 
displayed  in  a probability  diagram  for  hierarchical  inference* 
in  this  case*  for  a high  collection  effort.  The  probability 
diagram  in  Figure  5-8  shows  the  required  probabilities  for 
the  information  decision  diagram,  which  were  obtained  from 
the  diagram  in  Figure  5-7  by  using  Bayesian  hierarchical 
inference.  Once  this  procedure  has  been  followed  for  the 
medium  and  low  collection  efforts*  the  resulting  proba- 
biliLies  may  oe  inserted  in  the  original  information  decision 
diagram*  Figure  5-3,  and  the  diagram  may  then  be  folded  back 
to  determine  which  level  of  collection  effort  has  the  highest 
expected  value  and  is*  therefore*  the  best  course  of  action 
to  take. 

Folding  Back  the  Decision  Diagram 

For  convenience  in  presentation,  the  total  decisLon 
diagram  has  been  divided  into  three  sections*  each  repre- 
senting one  of  the  three  possible  levels  of  collection. 
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Figure  5-9  shows  the  branch  leading  from  the  left-most  act 
fork  and  representing  the  high  level  of  collection  Ch.  At 
the  right  are  the  posture  and  exposure  costs  for  each  out- 
come as  well  as  the  collection  costs  (which  are  higher  for 
the  less  extensive  evacuations  because  data  would  be  gathered 
over  longer  periods  of  time) . On  the  branches  from  the 
event  forks  are  the  probabilities  obtained  from  hierarchical 
inference.  Notice  that  the  listing  of  the  costs  is  not 
complete.  The  reason  is  that  for  a given  level  of  collec- 
tion, the  coats  are  a function  only  of  the  posture  assumed 
and  the  evacuation  situation,  and  so  repeat  exactly  for  the 
branches  representing  medium  and  low  indicators  of  activity. 

The  only  differenoea  between  these  branches  are  the  proba- 
bilities of  the  three  evacuation  situations.  First,  the 
costs  are  added  at  each  end  position  and  then,  according  to 
the  principles  described  in  Chapter  1,  expected  values  are 
computed  for  the  event  forks  leading  to  the  evacuation 
situations.  At  each  act  fork,  the  posture  which  has  the 
highest  expected  value  la  chosen,  as  indicated  by  blocking 
off  those  branches  which  are  not  chosen.  Finally,  the 
expected  value  for  the  high  collection  effort  is  obtained  by 
multiplying  the  probabilities  of  receiving  each  of  the  three 
kinds  of  information  by  the  expected  values  at  the  end  of 
those  three  branches.  In  this  case,  the  expected  cost  of 
the  high  collection  effort  is  -$13.64  million. 

In  exactly  the  same  fashion,  the  expected  value  is 
calculated  for  mounting  a medium  collection  effort  or  main- 
taining the  status  quo,  as  shown  by  the  diagrams  in  Figures 
5-10  and  5-11.  The  expected  values  for  the  two  collection 
efforts  are  -12.69  million  dollars  and  -12.26  million  dollars, 
respectively.  Thus,  the  best  course  of  action  is  for  CINCEUR 
to  continue  with  the  present  level  of  collection.  In  addi- 
tion to  identifying  which  course  of  action  has  the  highest 
expected  value,  the  decision  diagram  also  indicates  which 
posture  has  the  highest  expected  value  given  the  chosen 
collection  effort,  namely,  the  status  quo.  Figure  5-11 
shows  clearly  that  the  posture  decision  depends  on  the  kind 
of  information  received.  If  the  intelligence  source  indi- 
cates a high  level  of  activity,  then  the  course  of  action 
with  the  highest  expected  value  is  a high  readiness  posture, 

Ph.  This  conclusion  is  reached  by  following  the  I^  branch 
to  the  point  at  the  right  where  the  Ph  branch  was  chosen. 

If  the  information  indicates  a medium  or  low  level  of  activity, 
the  best  course  is  a medium  posture,  which  is  determined  by 
following  the  Im  and  the  II  branches  to  the  point  where  the 
Pm  branch  was  chosen. 

Value  of  Information 

Based  upon  the  expected  values  of  each  of  the  possible 
collection  efforts,  the  best  course  of  action  to  take  under 
these  conditions  is  to  continue  the  present  collection 
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effort.  , But  the  expected  values  for  alternative  collection 
efforts  di,tfer  for  two  reasons:  first,  because  the  infor- 
mation systems  differ  in  their  ability  to  report  accurately 
the  activity  level,  and  second,  because  they  differ  in  their 
costs.  Compared  to  the  status  quo,  the  additional  expected 
cost  of  a high  collection  effort  is  13.64  m^rus  12. or 
1.38  million  dollars,  while  the  extra  expected  cost  of  a 
medium  collection  effort  is  12.69  minus  12.26,  or  0.43 
million  dollars.  The  results  of  the  decision  analysis 
showed  that  one  should  continue  the  status  quo  collection 
effort  and  that  the  coats  of  the  medium  and  high  collection 
efforts  are  too  large  for  the  improved  accuracy  provided  by 
these  collection  systems.  However,  the  decision  analysis 
does  not  tell  how  much  too  large  these  costs  are. 

\n  alternative  way  to  analyze  the  information  decision 
is  to  discover  how  much  one  should  be  willing  to  pay  to 
obtain  the  information  capabilitius  represented  by  the  high 
and  medium  collection  efforts.  This  determination  can  be 
made  by  computing  the  expected  Values  for  each  alternative 
as  before,  without  including  the  cost  of  collection  in  the 
diagram.  Since  the  high  and  medium  collection  systems  are 
more  diagnostic  than  the  status  quo,  they  must  yield  a 
higher  expected  value  than  the  status  quo.  In  this  case,  the 
expected  value  of  the  high  ool3‘*ation  effort  is  -12.16 
million  dollars  and  the  expeett.  val^e  of  the  medium  collec- 
tion effort  is  -12.25  million  dollars.  Therefore,  since 
ClNCEUR's  expected  savings  are  -12.16  minus  -12.26  or 
$100,000  if  he  uses  the  high  collection  system,  and  -12.25 
minus  -12.26  or  $10,000  if  he  uses  the  medium  collection 
system,  he  should  be  willJxg  to  pay  these  amounts  respec- 
tively for  this  information.  Since  the  two  systems  actually 
cost  more  than  this,  he  will  save  money  by  not  buying  them. 

Notice  that  this  method  of  analysis  leads  to  the  same 
decision  as  before,  but  it  also  gives  additional  informa- 
tion. It  gives  the  decision  \ker  a good  idea  of  what  is 
required  of  an  information  sj-^tein  in  order  for  it  to  be  a 
good  buy  under  these  circumstances.  For  example,  suppose 
that  someone  offers  a third  information  system  which  hadn't 
been  previously  considered.  Rather  than  compute  the  entire 
decision  diagram  with  the  new  probabilities  and  costa,  it 
may  be  possible  to  determine  that  the  system  is  uneconomical 
without  making  any  calculations.  That  is,  unless  the  system 
is  substantial] more  diagnostic  of  the  evacuation  situation 
than  those  already  consiu  :teci  or  is  considerably  cheaper 
than  either  for  the  same  l^vel  of  diagnoaticity , it  will 
have  a lower  expected  valu(  than  the  status  quo. 

So  far,  for  purposes  of  illustration,  the  example 
involving  the  Cyprus  evacuatio*'  has  been  kept  fairly  simple. 
There  are  only  four  variables  two  decision  variables,  what 
level  of  collection  level  to  pursue  and  what  readiness 
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posture  to  adopt;  and  two  environmental  variables,  the  level 
of  information  received  and  the  true  evacuation  situation 
present.  In  this  particular  example,  each  of  these  variables 
can  take  on  three  values,  yielding  34,  or  81,  branches  of 
the  diagram.  However,  in  many  realistic  decision  situations, 
the  variables  will  often  take  on  more  than  three  values, 
with  a resulting  complexity  that  can  make  the  decision 
diagram  unmanageable.  Consider  what  would  happen  if  there 
were  ten  values  for  each  of  four  variables ; there  would  be 
lO**,  or  10,000,  branciiiis  for  the  four  variables.  Even  if 
the  variables  do  not  take  on  very  many  values,  a realistic 
problem  may  contain  many  variables.  If  an  analysis  con- 
tained six  variables  with  three  branches  leading  from  each, 
there  would  be  3®,  or  729,  branches  at  the  end  of  the  deci- 
sion diagram. 

Because  decision  diagrams,  especially  those  involving 
questions  about  the  value  of  information,  can  become  so 
complex,  a number  of  useful  analytical  shortcuts  have  been 
developed  to  simplify  the  task  of  analyzing  them.  One 
shortcut  is  to  compute  the  value  of  perfect  information. 

That  is,  suppose  you  could  obtain  information  from  a clair- 
voyant who  knew  which  evacuation  situation  was  taking  place. 
If  you  could  compute  the  value  of  this  information,  then 
this  value  would  place  an  upper  bound  on  the  value  of  any 
information  system:  since  this  is  perfect  information,  you 
. would  never  be  willing  to  pay  more  for  the  information 
output  of  any  real  system.  The  reason  for  constructing  and 
analyzing  a perfect  information  model  rather  than  the  one 
involving  the  actual  information  options  is  that  it  is 
easier  and  more  convenient  to  do  if  the  actual  information 
options  cost  more  than  perfect  information  is  worth. 

To  calculate  the  value  of  perfect  information,  a deci- 
sion diagram  (Figure  5-12)  which  resembles  the  other  decision 
diagrams  (except  that  the  chance  fork  representing  the  true 
evacuation  situation  can  be  eliminated)  is  used.  This 
situation  is  omitted  because  once  the  decision  maker  has 
received  information  from  the  clairvoyant  (that  is,  when  a 
particular  bra:.*ch  from  the  first  chance  fork  has  been  taken)  , 
there  is  no  remaining  uncertainty  about  the  true  evacuation 
situation.  Accordingly,  no  assessments  of  probabilities  are 
necessary  for  the  true  evacuation  situation.  In  addition, 
since  the  clairvoyant  will  provide  the  information  that  the 
situation  requiring  maximum  evacuation  is  occurring  whenever 
the  situation  is  in  fact  occurring,  the  probability  of  the 
decision  maker's  receiving  that  information  is  equal  to  the 
probability  of  the  situation  occurring,  or  the  prior  proba- 
bility. Figure  5-12  shows  that  the  expected  value  of  perfect 
information  is  -10.10  million  dollars.  Since  the  expected 
value  of  the  status  quo  or  low  collection  effort  is  -12.26 
million  dollars,  the  value  of  perfect  information  is  -10.10  - 
(-12.26),  or  2.16  million  dollars. 


141 


Ph 


-176.87M 
-233.50M 
-261. 06M 

-B.48M 

-2.28M 

-3.13M 

-4.8gM 

-0.06M 

0 


DECISION  DIAGRAM  FOR 
PERFECT  (CLAIRVOYANT)  INFORMATION 


It  is  apparent  that,  in  this  example,  perfect  informa- 
tion puts  only  a weak  upper  bound  on  the  value  of  informa- 
tion; the  value  of  perfect  information  is  2.16  million 
dollars,  which  is  considerably  more  than  either  the  high  or 
the  medium  collection  effort  costs.  However,  it  is  important 
to  remember  that  the  information  system  was  part  of  a hier- 
archical inference. ' The  information  system  was  not  designed 
to  collect  information  directly  about  the  evacuation  situa- 
tion but  rather  about  activity  level.  Since  activity  level 
is  not  perfectly  (.'orrelated  with  the  evacuation  situation, 
no  information  sys  -.em  which  gathers  information  about  activity 
level  can  be  expected  to  predict  the  evacuation  perfectly. 

This  fact  suggests  a stronger  upper  bound,  the  value  of  a 
perfect  report  about  activity  level.  Figure  5-13  shows  the 
decision  diagram  for  the  calculation  of  the  expected  value 
of  a perfect  report  about  activity  level.  The  only  dif- 
ference between  this  diagram  and  the  decision  diagrams  for 
the  high  and  the  medium  collection  efforts  i<  that  the 
probabilities  are  derived  from  a simple  sing'. e-stage  Bayesian 
inference  rather  than  from  a hierarchical  Bayesian  inference. 
Specifically,  the  probabilities  can  be  calculated  by  applying 
Bayes'  Theorem  to  the  probabilities  in  the  upper  matrix  of 
Figure  5-6.  The  result  of  folding  back  this  decision  diagram 
is  that  the  expected  value  of  a perfect  report  about  activity 
level  is  found  to  be  -11.98  million  dollars.  Compared  to 
the  expected  value  of  the  status  quo  collection  effort  of 
-12.26  million  dollars,  the  value  of  a perfect  report  about 
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activity  level  is  $320,000.  This  means  that  no  information 
system  which  reports  about  activity  level,  even  if  it  gave  a 
perfect  report,  is  worth  inore  than  $320,000.  Recall  that 
the  moderate  collection  effort  has  an  expected  additional 
cost  of  0.43  million  dollars  over  the  low  collection  effort, 
and  the  expected  increment  for  the  high  collection  effort  is 
1.38  million  dollars.  Since  both  are  considerably  less  than 
perfect  in  reporting  about  activity  level,  it  is  obvious 
without  constructing  decision  die\gram8  for  these  two  levels 
of  information  collection  that  they  cost  more  than  they  are 
worth  in  terms  of  reducing  the  expected  costs  of  the  posture 
decision. 


Figure  5-14  sunucnarizes  the  value  of  inforr.ation  at  low, 
medium,  and  high  collection  levels  as  well  as  the  two  upper 
bounds  on  information  value,  a perfect  report  about  activity 
level  and  clairvoyance. 
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Value  Diagrams 

As  was  pointed  out  earlier,  decisions  to  acquire  infor- 
mation that  has  the  potential  of  improving  a primary  deci- 
sion are  called  "information  decisions."  Although  the 
primary  decision  drives  an  information  decision,  the  infor- 
mation decision  must  occur  before  the  primary  decision  is 
actually  made  in  order  that  the  acquired  information  has  a 
possibility  of  influencing  the  primary  decision. 

As  described  in  the  preceding  section,  decision- theo- 
retic analysis  of  an  information  decision  includes  four 
distinguishable  events.  First,  the  information  decision 
whether  to  purchase  information  and,  if  so,  what  kind  and 
how  much  is  made;  second,  the  information  is  acquired; 
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third,  based  upon  the  outcome  of  the  information  acquisi- 
tion, the  primary  decision  is  made;  and  finally,  some  state 
of  nature  occurs  which,  together  with  the  decision  made, 
results  in  a parti.c\}\:‘:c  outcome  which  can  be  evaluated  by 
the  decision  maker  " u general  principle  is  that  infor- 
mation is  purchas-^v.  if  it  has  a potential  for  improving 

the  expected  value  of  a primary  decision  by  an  amount  that 
is  greater  than  the  cost  of  collecting  the  information. 

One  important  class  of  information  decisions  involves 
collecting  information  for  intelligence  agencies:  what  type 
and  how  many  platforms  should  be  used,  should  a new  expen- 
sive collection  system  be  purchased,  how  can  the  collection 
budget  be  cut  substantially  without  doing  serious  damage  to 
the  amount  of  valuable  information  being  collected,  and  the 
like?  In  these  situations  it  is  difficult  if  not  impossible 
to  apply  the  usual  techniques  of  information  decisions.  One 
of  the  problems  is  that  the  primary  decisions  to  be  made  in 
the  future  are  not  known  presently,  and  even  if  they  were 
known,  it  would  be  difficult  to  judge  the  impact  that  a 
particular  item  of  information  would  have  on  them.  Another 
problem  is  that  the  person  who  makes  the  information  decision 
may  not  possess  the  necessary  data  to  make  the  primary 
decision.  For  example,  the  manager  in  charge  of  allocating 
collection  platforms  may  not  be,  and  usually  is  not,  respon- 
sible for  the  determination  of  U.S.  foreign  policy.  Although 
this  separation  of  functions  may  not  be  an  ideal  state  of 
affairs  from  a conventional  decision- theory  standpoint,  it 
is  necessary  for  decision  theory  to  provide  a methodology 
that  will  operate  within  real-world  conetraints.  Therefore, 
even  though  it  is  useful  formally  to  consider  the  role  of 
primary  decisions  as  deteinniners  of  information  decisions, 
it  is  also  important  to  consider  procedures  for  measuring 
the  value  of  information  without  explicitly  evaluating  its 
impact  on  primary  decisions. 

Direct  Assessment  of  Value  - Value  Diagreuns 

One  theoretical  model  which  is  used  to  measure  the 
value  of  information  in  situations  in  which  it  is  inappro- 
priate to  consider  primary  decisions  explicitly  is  called  a 
value  diagram  (this  approach  has  also  been  called  assessment 
of  goal-dependent  utilities  and  goal-fabric  analysis.) 

The  basic  concept  of  a value  diagram  is  the  construc- 
tion of  a hierarchical  structure  in  such  a way  that  it  is 
possible  to  measure  or  assess  the  relative  contribution  of 
different  collection  systems  to  the  overall  goals  of  collec- 
tion. This  structure  is  achieved  by  decomposing  the  problem 
so  that  it  is  possible  to  assess  the  value  of  information 
that  each  of  the  collection  systems  contributes  to  each  of 
the  several  subgoals  and  then  to  assess  the  importance  of 
the  subgoals  with  respect  to  the  major  goals. 
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Figure'  5-15  is  a schematic  diagram  of  an  actual  analysis 
illustrating  how  a value  diagram  works.  The  goal  of  the 
analysis  was  to  assess  the  relative  value 
being  collected  by  several  different,  separately  fundable 
collection  systems  or  platforms,  with  a particular  emphasis 
on  evaluating  the  air  platforms.  The  ultimate  purpose  of 
the  analysis  was  to  reduce  the  current  budget  for  air 
lection  in  such  a manner  that  it  would  have  a minimal  effect 
on  the  value  of  information  collected. 
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PARTIAL  VALUE  DIAGRAM 
FOR  INTELLIGENCE  COLLECTION  PLATFORMS 


The  left-hand  side  of  Figure  5-15  indicates  the  goals 
and  subgoals  of  the  information  collection  and  the  right- 
hand  aide  indicates  the  disciplines  and  platforms  which  are 
responsible  for  collecting  information  relevant  to  the  goals 
and  aubgoals.  The  numbers  on  each  branch  are  importance 
weights  representing  the  relative  importance  of  each  branch 
subgoal  to  the  subgoal  or  goal  immediately  to  the  left.  At 
each  fork  the  importance  weights  on  all  the  branches  are 
constrained  to  sum  to  1.0.  The  importance  weights  on  the 
left  indicate  the  relative  importance  of  information  from 
different  geographical  regions,  about  differing  threats,  and 
about  various  types  of  targets.  On  the  right  side,  the 
weights  indicate  the  assessed  relative  values  of  information 
being  collected  by  the  different  disciplines  about  each 
target  type,  and  then  the  importance  of  each  system  within 
each  discipline. 
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Consider  first  the  geographic  regions.  The  weights 
attached  to  the  branches  imply  that  it  is  twice  as  important 
to  obtain  information  in  region  A as  in  region  C and  four 
times  as  important  to  obtain  information  in  region  A as  in 
region  D.  The  next  branch  indicates  that>  within  region 
information  about  strategic  threats  is  four  times  as  impor- 
tant as  information  about  tactical  threats.  By  contrast « 
within  geographic  region  D,  information  about  tactical 
threats  is  assessed  to  be  as  important  as  information  about 
strategic  threats.  Similarly,  the  relative  importance  of 
strategic  and  tactical  threat  information  was  also  assessed 
for  geographic  regions  A and  C.  However,  rather  than  dis- 
playing the  complete  analysis,  the  diagram  in  Figure  5-15 
illustrates  the  way  the  analyst  proceeds  in  constructing  the 
diagram. 

Of  the  tactical  information  collected  in  region  B, 
information  about  ground  forces  is  the  most  important,  and 
information  about  air  forces  is  more  important  than  that 
about  naval  and  missile  forces,  which  are  equally  important. 

The  collection  part  of  the  diagram  is  divided  into 
disciplines  and  platforms.  The  figure  shows  that  the  value 
of  photographic  intelligence  currently  being  collected 
against  naval  tactical  forces  in  geographic  region  B is 
eight  times  as  great  as  the  value  of  information  being 
collected  through  the  intercept  of  communications,  and  ten 
times  as  important  as  information  being  collected  from  radar 
intercepts  in  that  area.  Finally,  information  which  is 
provided  by  attaches  has  been  assessed  to  have  the  least 
value  for  naval  tactical  forces  in  region  B.  The  figure 
also  shows  that  90%  of  the  value  of  photographic  information 
is  currently  being  obtained  by  airborne  collection  platforms 
and  10%  by  systems  which  are  not  airborne.  Of  the  airborne 
platforms,  aircraft  A contributes  50%  of  the  total  value  of 
the  information  collected,  and  the  combination  of  aircraft  D 
and  E contributes  the  remaining  50%.  For  informtion  ob- 
tained from  communications  intercepts,  only  40%  comes  from 
airborne  platforms,  whereas  70%  of  the  value  of  this  infor- 
mation is  obtained  from  airplane  B.  Notice  that  aircraft  A 
contributes  both  photographic  and  communloationa  information 
whereas  B and  C contribute  only  communication  information 
and  aircrafts  D and  E contribute  only  photographic  infor- 
mation. 

When  all  of  the  value  assessments  have  been  made  for 
all  of  the  branches  of  the  complete  value  diagram,  it  can  be 
folded  back  in  exactly  the  same  way  as  a probability  diagram. 
The  relative  value  of  a given  platform  is  a weighted  sum 
equal  to  the  value  of  the  information  contributed  to  a goal 
or  subgoal  weighted  by  the  relative  importance  of  those 
goals  and  summed  across  all  goals.  For  example,  using  the 
values  from  Figure  5-15,  we  can  compute  the  relative  value 
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of  the  photographic  information  collected  by  air  platform  A 
about  tactical  naval  targets  in  geographic  region  B to  be 
equal  to  the  products  of  all  the  weights  on  that  path  (0.30 
X 0.20  X 0.10  X 0.80  X 0.90  x 0.50  > 0.00216). 

By  proceeding  in  this  manner  for  all  paths  through  the 
complete  diagram,  we  can  obtain  the  simplified  value  diagram 
shown  In  Figure  5-16.  This  diagram  shows  that  air  platform 
A contributes  12%  of  the  total  value  of  all  Information  from 
all  collection  systems  about  all  threats  in  all  geographic 
areas.  It  also  shows  that  all  the  air  platforms  together 
contribute  30%  of  the  value  and  "not  air"  sources  contribute 
70%. 


AIR 


NOT  AIR 


A 

INFORMATION 

VALUE 

.120 

B 

_ .090 

» •• 

C 

.030 

MU* 

D 

_ .045 

E 

.015 

.700 

Figure  5-16 

SOLUTION  OF  COMPLETE  VALUE  DIAGRAM 


Methods  of  Asaesslnq  Importance  Weights 

There  are  three  possible  methods  of  assigning  importance 
weights  to  the  branches  of  the  value  diagram.  While  In 
theory  there  should  be  no  difference  in  the  result  obtained 
by  the  three  methods,  in  practice  the  three  methods  usually 
do  differ.  Unfortunately,  research  has  not  yet  determined 
which  of  the  three  methods  is  most  accurate.  Therefore,  it 
is  a good  practice  to  use  al]  three  methods  and  resolve  any 
discrepancies  that  appear  among  them. 

The  first  method  is  to  distribute  100  points  across  all 
the  alternatives.  The  advantage  of  this  method  is  that  it 
resembles  the  actual  resource  allocation  problem.  For 
example,  the  assessor.  Imagining  that  he  had  $100  million, 
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might  wish  to  spend  $25  million  on  the  Soviet  Union,  $10 
million  on  East  Germany,  and  so  on.  The  disadvantage  of  the 
method  is  that  the  resulting  distribution  curve  of  money 
(points)  is  often  too  flat.  This  distribution  often  results 
if  too  many  alternatives  are  considered,  or  if  too  great  a 
discrepancy  exists  between  the  largest  and  smallest.  In  the 
above  example,  if  the  analyst  assigns  $28  million,  or  28  of 
the  100  points,  to  Russia,  it  might  be  necessary  for  him  to 
assign  .003  points  to  Ghana  in  order  that  the  relative  sizes 
be  kept  accurate.  Since  analysts  generally  will  not  assign 
less  than  1 point  to  any  country,  the  consideration  of  many 
little  countries  can  result  in  insufficient  points  remaining 
to  give  enough  to  the  more  important  countries. 

The  second  method  is  to  use  relative  magnitudes.  In 
this  method  the  most  Important  branch  is  assigned  100  points 
and  the  other  branches  are  assigned  points  according  to 
their  importance  relative  to  this  branch.  For  example,  if 
one  branch  has  one-half  the  value  of  the  most  important, 
it  would  receive  50  points.  Research  has  shown  that  this 
method  also  tends  to  result  in  a uniform  distribution 
apparently  because  experts  are  reluctant  to  assign  small 
values. 

The  third  method  of  assessing  importance  weights  is  to 
use  a ratio;  typically,  the  moat  important  factor  is  com- 
pared to  each  of  the  other  factors  and  a direct  ratio  is 
assessed  by  the  expert.  For  example,  the  expert  might  judge 
that  the  most  important  factor  had  five  times  the  value  of* 
the  least  important  but  only  three  times  the  value  of 
another  factor  and  so  on.  Research  has  shown  that  this 
method  generates  a more  extreme  distribution,  but  it  is  not 
known  whether  the  results  are  more  accurate.  However,  it 
is  a particularly  valuable  form  of  feedback  when  one  of  the 
other  methods  has  been  used  initially.  One  possible  dis- 
advantage to  this  method  is  that  a relatively  small  error  in 
estimation  of  the  ratio  may  result  in  a very  large  error  in 
the  final  value  of  information.  For  example,  if  an  expert 
assessed  the  importance  of  platform  1 to  country  A to  be  3:1 
rather  than  2:1,  and  the  Importance  of  country  A to  the 
European  area  to  be  ^:1  instead  of  4:1,  the  resulting  rela- 
tive importance  of  platform  1 to  the  European  region  would 
be  15:1  instead  of  8:1.  In  contrast,  the  distribution  of 
points  would  have  been  75  and  25  instead  of  67  and  33  in  the 
first  assessment,  and  83  and  17  instead  of  80  and  20  in  the 
second  assessment,  for  the  same  error  to  occur.  Another 
possible  problem  is  that  the  expert  may  jump  from  a ratio  of 
1:1  to  2:1  unless  he  is  encouraged  to  use  ratios  such  as 
3:2,  5:4,  and  so  on. 

Once  importance  weights  have  been  assigned  to  each  of 
the  forks  in  the  value  diagram,  the  diagram  may  be  folded 
back  in  exactly  the  same  manner  as  a probability  diagram. 
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Thus,  just  as  solving  a probability  diagram  yields  an  average 
probability  or  an  expected  probability,  folding  back  a value 
diagram  yields  a weighted  value.  If  the  ratio  method  has 
been  used  to  check  the  values  or  to  obtain  them  originally, 
the  final  scale  should  have  the  properties  of  a ratio  scale. 
That  Is,  It  should  have  a meaningful  zero  (i.e.,  a zero  cm 
the  information  value  scale  means  zero  value,  and  a weighted 
value  of  6 should  be  twice  as  Important  as  a value  of  3) . 

Once  such  a scale  has  been  obtained,  it  Is  possible  to  use 
the  value  of  information  to  specify  the  allocation  of  re- 
sources . 

Resource  Allocation 


One  way  in  which  a decision  maker  might  allocate  re- 
sources is  to  distribute  money  proportionally  to  the  value 
of  information;  that  is,  a source  of  information  which  has 
28%  of  the  value  would  receive  28%  of  the  resources.  Another 
method  is  to  use  a lexicographic  decision  rule.  The  infor- 
mation sources  are  listed  in  decreasing  order  of  value  of 
information.  The  resources  are  allocated  to  the  sources  by 
starting  at  the  top  of  the  list  and  proceeding  down  it  until 
all  available  resources  have  been  allocated.  The  problem 
with  both  of  these  methods  is  that  they  make  an  implicit 
assumption  that  information  is  equally  costly  for  any  of  the 
sources,  an  assumption  which  is  practically  never  true.  In 
the  best  of  all  possible  worlds,  the  source  which  has  the 
best  information  would  be  the  cheapest  to  operate.  In  this 
case,  one  would  purchase  as  much  of  that  kind  of  information 
as  possible,  then  as  much  of  the  next  most  valuable,  and  so 
on  until  the  resources  have  been  expended.  In  the  worst 
case,  which  may  occasionally  occur,  those  sources  that  pro- 
vide the  most  valuable  information  are  the  most  costly. 
Accordingly,  the  decision  maker  must  make  a trade-off  between 
the  cost  of  information  and  the  v^lue  of  that  information. 

One  way  of  proceeding  ir  this  situation  is  to  describe 
each  information  source  in  terms  of  a value-to-cost  ratio. 

As  a first  approximation,  the  optimum  allocation  policy  can 
be  approached  by  ordering  these  information  sources  lexi- 
cographically according  to  decreased  value- to-cost  ratio.  The 
resulting  decision  rule  says  to  start  buying  at  the  top  and 
to  continue  to  buy  sources  until  the  budget  ceiling  is 
reached.  In  some  cases,  however,  this  approach  will  be 
slightly  suboptimum  because  it  is  possible  that  a combination 
of  several  information  sources,  each  with  a lower  value-to-cost 
ratio,  will  give  a slightly  higher  aggregate  value  than  one 
inform.ation  source  with  a slightly  larger  ratio.  If  the 
number  of  possible  sources  is  small,  it  is  possible  to 
obtain  the  optimum  solution  by  merely  listing  all  combina- 
tions of  sources  and  then  selecting  the  combination  which 
has  the  largest  value  for  a total  cost  less  than  the  budget 
ceiling.  Unfortunately,  since  for  even  a modest  number  of 


sources,  the  total  number  of  combinations  is  astronomical, 
a straightforward  application  of  this  algorithm  is  impractical. 
The  optimum  solution  of  the  situation  can  be  found  by  using 
integer  programming  techniques.  Integer  programming  is  a 
computational  methodology  which  maximizes  one  variable,  in 
this  case,  information  value,  subject  to  a constraint  on 
another  variable,  in  this  case,  cost. 

In  many  applications,  it  is  not  merely  a matter  of  de- 
ciding whether  or  not  to  use  a particular  source  of  informa- 
tion, but  rather  how  much  information  to  acquire  from  each 
source.  In  this  case,  it  is  not  possible  to  describe  the 
source  of  information  in  terms  of  a single  ratio  relating 
value  to  coat.  In  general,  the  value  of  a piece  of  infor- 
mation, i,  is  not  the  same  as  the  value  of  the  piece  of 
information  (i-1)  that  preceded  it.  It  is  often  the  case 
that  the  relationship  of  information  value  to  cost  is  a 
monotonically  increasing,  negatively  accelerated  function. 

That  is,  the  more  information  you  receive,  the  more  total 
value  it  has,  but  each  increase  in  information,  for  a given 
increment  of  cost,  is  worth  less  than  the  previous  increase 
for  the  same  increment  of  cost.  Under  these  conditions  the 
optimum  allocation  of  resources  can  be  found  by  the  following 
graphic  technique: 

First,  a curve  relating  information  value  to  cost  for 
each  source  is  drawn.  Then,  on  the  graphs  relating  value 
to  coat  for  each  source,  a very  high  ratio  of  value  to  cost 
is  set,  and  a straight  line  with  that  slope  is  drawn  through 
the  origin.  The  resulting  cost  of  this  allocation  will  be 
the  sum  of  costs  associated  with  each  of  the  points  at  which 
this  "criterion"  line  Intersects  each  of  the  curves.  (Because 
of  the  properties  of  the  function,  a straight  line  from  the 
origin  through  any  point  on  the  curve  to  the  left  of  the 
"criterion  line"  intersection  will  have  a slope  greater  than 
the  "criterion  line"  and,  therefore,  a greater  value-to- 
cost  ratio.)  If  the  initial  choice  for  the  "criterion  line" 
slope  represents  a sufficiently  high  value-to-cost  ratio, 
the  total  cost  obtained  v/ill  be  less  than  the  cost  ceiling. 

The  process  is  then  repeated,  successively  decreasing  the 
slope  of  the  criterion  line.  When  the  slope  (value-to- 
cost  ratio)  of  a chosen  line  tosults  in  costs  for  each 
source,  derived  from  the  intersection  point  on  each  curve, 
which  add  up  to  the  coat  ceiling,  an  optimal  resource  allo- 
cation will  have  been  achieved  for  that  cost  constraint. 

This  technique  works  for  any  monotonically  increasing 
function  which  has  no  inflection  point.  However,  many 
actual  collection  systems  exhibit  an  S-shaped  function. 

This  function  occurs  because,  at  a low  level  of  collection, 
much  information  obtained  is  not  identified  or  correlated 
with  previous  information  and  therefore  has  very  low  infor- 
mation value.  Once  a certain  amount  has  been  collected. 
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however#  the  information  begins  to  fit  together  in  a pattern 
of  significant  interest.  At  this  point,  any  subsequent 
increase  in  the  level  of  collection  results  in  a large 
increase  in  value  until  finally  the  system  begins  to  approach 
the  physical  limits  of  information  it  can  collect,  at  which 
point  the  value- to-cost  ratio  levels  off  again.  In  any 
function  which  has  an  inflection  point,  the  "criterion  line" 
may  intersect  the  curve  in  more  than  one  place;  it  is  not, 
therefore,  a simple  matter  to  determine  the  optimum  alloca- 
tion of  resources.  In  this  case,  the  usual  procedure  is  to 
try  several  alternative  allocations  in  order  to  reduce  the 
chance  of  a local  maximum  being  found.  A more  systematic 
approach,  however,  employs  a general,  non-linear  programming 
algorithm,  which  will  nearly  always  produce  the  optimum  solu- 
tion. 
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CHAPTER  6 


APPLICATION  OF  DECISION-ANALYTIC  METHODS:  A CASE  STUDY 


This  chapter  presents  a case  study  o£  a decision  analy- 
sis as  an  illustration  of  the  way  in  which  decision  theory 
can  be  used  to  clarify  a problem  involving  uncertainty  and 
complex  value  judgments  and  thxis  help  the  decision  maker  to 
determine  which  of  several  courses  of  action  is  the  best  one 
to  take.  The  example  is  simple#  but  it  is  extended  in  many 
directions  which  are  often  of  interest  to  a decisivon  maker. 
Very  few  real  problems  will  be  this  simple,  nor  will  they 
investigate  so  many  avenues;  typically,  a decision  maker  is 
interested  in  a detailed  application  of  one  or  two  aspects 
of  decision  analysis  rather  than  a more  superficial  applica- 
tion of  many. 

Within  these  restrictions,  the  analysis  of  the  case 
study  is  developed  in  a logical  fashion,  much  as  it  might  be 
in  a real  decision  problem.  In  this  process,  we  use  many  of 
the  concepts  introduced  in  the  previous  chapters:  decision 
tree,  probability,  subjective  value,  expected  value,  revision 
of  probabilities,  and  the  value  of  information.  In  addition 
to  illustrating  how  these  concepts  can  be  applied,  we  also 
want  to  show  how  a decision  analysis  might  proceed. 

The  case  study  involves  a problem  faced  by  the  com- 
mander of  a small  naval  task  force  in  the  scenario  depicted 
in  Figure  6-1.  One  of  his  ships  has  been  attacked  by  an 
enemy  airplane,  and  it  has  sustained  deunagc  and  casualties. 
However,  this  is  the  first  time  such  an  attack  has  ever 
occurred,  and  the  damaged  ship,  UES  HERRINGBONE,  did  not 
fire  at  the  attacking  aircraft,  although  it  had  been  given 
permission  to  fire  at  will.  The  task  force  commander, 
anticipating  that  such  an  attack  might  occur  again  in  the 
future,  wants  to  know  whether  the  captain  of  the  HERRINGBONE 
made  the  correct  decision,  given  his  uncertainty  about  the 
identity  of  the  approaching  aircraft.  More  generally,  he 
would  like  to  know  if  additional  information  would  have 
resulted  in  a better  decision,  and  how  much  that  information 
would  be  worth.  To  help  him  answer  these  questions,  he  has 
called  in  a consultant  who  is  expert  in  applied  decision 
analysis. 

What  follows  is  a dialogue  between  the  task  force 
commander  and  the  expert  consultant,  in  which  they  try  to 
arrive  at  a formal  representation  of  the  problem,  and,  from 
that,  the  solution.  Notice  that  as  the  problem  develops,  it 
is  necessary  for  the  consultant  to  learn  to  understand  the 
captain's  view  of  the  problem  as  well  as  to  understand  the 
problem  itself.  Unless  the  consultant  can  understand  the 
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Figure  6-1 


motivations  of  the  decision  maker  as  well  as  the  objective 
characteristics  of  the  problem,  he  cannot  really  make  full 
use  of  the  tools  of  decision  theory. 

Structure  :"if  the  Decision 

Commander:  Let  me  describe  the  problem.  Imagine  that 
I have  just  picked  up  an  unidentified  aircraft  on  radar,  and 
it  is  coming  straight  for  my  ship.  It  could  be  an  enemy 
plane  attacking,  but  it  could  also  be  one  of  our  aircraft 
which  has  been  shot  up  and  is  ditching  in  the  water  near  my 
ship  to  be  rescued.  Now,  of  course,  I have  various  kinds  of 
information  about  whether  it  Is  friendly  or  not,  but  I can 
never  be  absolutely  sure,  and  I may  not  have  time  to  look  at 
my  information  before  1 have  to  make  a decision.  I'd  like 
to  analyze  this  situation  now  so  I can  make  a more  informed 
decision  if  the  problem  ever  occurs  again.  How  do  we  start? 

Consultant:  I think  the  way  to  begin  structuring  the 
problem  is  to  use  a simple  decision  diagram  (see  Figure  6- 
2) . We  begin  on  the  left,  drawing  lines  for  the  options 
available  to  you  and  labeling  them.  This  is  called  an  act 
fork.  In  this  case,  there  are  only  two  available  courses  of 
action,  SHOOT  and  NO  SHOOT.  Now,  what  will  happen  if  you 
shoot  at  the  airplane? 

Commander:  Well,  you  can  either  knock  him  out  of  the 
air,  or  you  can  miss  him.  Of  course,  our  crews  are  trying 
their  beat  to  bring  him  down.  And  sometime  during  or  after 
the  action,  we  will  find  out  that  the  airplane  is  a friend, 
or  that  it  is  an  enemy. 

Consultant:  We'll  represent  the  first  set  of  possi- 
bilities as  KILL  or  NO  KILL,  the  second  as  FRIEND  or  ENEMY. 
In  neither  case  are  the  possibilities  entirely  under  your 
control,  so  we  call  them  events  and  represent  them  by  event 
forks . To  distinguish  the  act  forks  from  the  event  forks,  I 
will  use  a box  for  the  act  forks,  and  a circle  for  the  event 
forks.  In  drawing  the  event  forks  on  the  decision  tree,  I 
would  normally  show  them  in  the  order  that  the  events  will 
become  known  to  the  decision  maker — shoot  first,  then  learn 
the  identity.  But  there  are  exceptions  to  that  rule,  and 
this  may  be  one.  It  seems  clear  to  me  that  the  chances  of 
the  airplane  being  a friend  or  an  enemy  are  the  same  whether 
you  score  \ hit  or  not,  but  isn't  it  possible  that  the 
chances  of  a hit  might  be  affected  by  the  aircraft  being  a 
friend  or  enemy? 

Commander:  I suppose  that  could  be  so;  some  types  of 
airplanes  are  slightly  more  difficult  to  hit  then  others. 

Consultant;  Then  to  be  safe,  we'd  better  show  KILL  or 
NO  KILL  as  following  FRIEND  or  ENEMY.  So,  we  will  put 
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FRIEND  or  ENEMY  event  forks  at  the  end  of  each  of  the  initial 
acts  branches  and,  then,  in  the  case  of  shooting,  follow 
these  with  KILL  or  NO  KILL  event  forks  (see  Figure  6' 2).  I 
assume  that  if  an  enemy  gets  through,  he  will  try  to  bomb 
the  ship. 

Commander:  That's  right,  but  he  may  not  score  a hit. 

Consultant:  So,  to  complete  the  tree,  we  will  add  an 
event  fork,  with  branches  HIT  and  NO  HIT,  at  the  end  of  the 
path  SHOOT,  ENEMY,  NO  KILL,  and  the  same  fork  after  NO 
SHOOT,  ENEMY  (see  Figure  6-’2).  Now,  it  is  obvious  that  we 
can  extend  this  structure  in  many  ways,  but  at  this  point  I 
think  we  have  the  basis  for  exploring  the  decision  when  we 
incorporate  your  opinions  and  values. 

The  first  task  is  to  decide  what  the  outcomes  are,  and 
what  attributes  are  important  for  assessing  the  value  of 
those  outcomes.  The  first  outcome.  A,  is  that  you  shoot  at 
a friendly  aircraft  and  hit  it.  What  are  the  important 
things  about  this  outcome? 


FRIEKO  ; 


NO  KILL 


SHOOT 


ENEMY 


NO  KILL 


DECISION 


NO  HIT 


Figure  6-2 


Conmiander;  The  most  important  thing  is,  of  course,  the 
life  of  the  pilot.  Second  to  that  comes  the  value  of  the 
airplane. 

Consultant:  O.K.,  that  seems  reasonable,  but  when  you 
say  the  life  of  the  pilot,  how  do  you  mean  that?  Do  you 
mean  the  effect  that  killing  one  of  your  own  aviators  has  on 
the  U.S.  public,  or  on  the  morale  of  the  men  in  the  convoy, 
or  do  you  mean  the  dollar  value  that  the  loss  of  an  aviator 
means  to  the  war  effort? 

Commander:  Well,  I hadn't  really  thought  about  it  that 
way,  but  1 guess  I mean  all  three.  I'm  certain  that  all 
three  of  these  enter  into  my  decision  to  shoot  or  not  shoot. 

Consultant : Then  we ' re  going  to  have  to  make  some 
assumptions  or  the  decision  diagram  may  become  unmanageable. 
Let's  assume  that  if  you  hit  the  aircraft  that  you  kill  the 
pilot.  Also,  we  have  to  assume  a particular  type  of  U.S. 
airplane]  let's  say  an  A-4.  What  are  the  consequences  if 
you  miss  him?  That's  outcome  D. 

Commander:  Nothing  happens.  Well,  you  lose  a missile, 
or  a shell,  but  they're  expendable,  so  they  don't  really 
count. 

Consultant:  At  outcome  C,  you  shoot  down  an  enemy  air" 
plane.  Again,  we  assume  you  kill  the  pilot,  and  again  we 
will  assume  a particular  enemy  aircraft.  What  kind  of  plane 
did  the  enemy  use  when  they  attacked  the  HERRINGBONE? 

Commander:  They  used  an  old  MIG-17  that  they  get  from 
the  Soviets. 

Consultant:  What  about  outcome  D?  In  this  case,  the 
enemy  aircraft  gets  through,  bombs  the  ship,  and  scores  a 
hit.  What  damage  would  you  say  would  be  sustained? 

Commander:  Well,  the  HERRINGBONE  lost  eight  men,  but 
it  was  an  unusually  light  hit.  I would  think  that,  in 
general,  about  25  men,  perhaps  3 officers  and  22  enlisted 
men,  would  be  lost  in  this  kind  of  attack.  And,  of  course, 
the  ship  would  have  to  be  dry-docked  for  repairs. 

Consultant:  Now  let's  move  on  to  outcome  E;  the  enemy 
fails  to  hit  the  ship.  In  this  case,  he  quickly  leaves  the 
scene,  so  nothing  further  happens.  At  outcome  F,  you  don't 
shoot  at  a friend,  so  nothing  happens.  Next,  at  outcome  G, 
your  ship  is  hit  because  you  failed  to  shoot  at  the  enemy 
aircraft  and  his  bombing  run  was  successful  from  his  point 
of  view.  It  looks  like  that  outcome  is  almost  the  same  as 
D.  Finally,  outcome  H is  like  outcome  E. 
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value  of  Outcomes 


In  order  to  structure  any  decision  problem,  the  person 
making  the  decision  must  assess  how  much  each  of  the  possible 
outcomes  is  worth  to  him.  As  we  explained  in  Chapter  2, 
this  worth  is  called  the  utility  of  the  outcome,  and  when  it 
is  cc  .josed  of  several  attributes  or  dimensions,  it  is 
called’ multi-attribute  utility.  One  strategy  here  is  to 
assess  how  much  each  outcome  is  worth  for  each  attribute  and 
how  important  each  attribute  is,  then  take  the  values  from 
each  of  the  attributes,  weight  them  by  the  importance  of  the 
attribute,  and  combine  them  into  an  overall  utility.  This 
is  the  procedure  followed  by  the  consultant  in  this  case.^ 

Consultant:  Now  we  have  to  discover  just  how  much  each 
of  these  possibilities  is  worth  to  you,  the  captain  of  the 
attacked  ship.  To  do  this,  we  will  use  each  of  the  four 
attributes  we  have  talked  about:  men,  equipment,  political 
value,  and  morale.  Let's  start  with  the  MEN  attribute. 
Probably  it  is  easiest  to  compare  the  value  of  one  U.S. 
pilot  to  the  value  of  one  enemy  pilot. 

Commander:  My  first  impulse  is  to  say  that  U.S.  pilots 
are  more  valuable  because  they  are  better  pilots  than  the 
enemy,  and  it  costs  us  more  to  train  them  than  it  does  the 
enemy.  On  the  other  hand,  we  have  a much  larger  supply  of 
them  than  they  do,  and  we  can  produce  them  in  less  time. 

Consultant:  O.K.,  I want  you  to  think  aloud  and  fill 
in  the  details  of  these  considerations.  The  details  them- 
selves aren't  too  important,  but  as  you  think  through  the 
concrete  facts,  you  should  be  more  able  to  come  up  with  a 
consistent  assessment  of  the  relative  worth  of  the  two 
aviators. 

Commander:  First  of  all,  the  U.S.  produces  5,000 
pilots  a year,  at  a cost  of  about  $1  million  each,  and  it 
takes  approximately  18  months  to  train  them.  The  enemy, 
however,  probably  produces  less  than  100  pilots  a year  at  a 
cost  of  perhaps  $100  thousand  each,  and  it  takes  about  three 
years  to  train  them.  Also,  it  is  important  to  remember  that 


^However,  it  should  be  pointed  out  that  because  direct  assess- 
ments of  worth  are  made,  the  commander's  attitude  toward  risk 
is  not  accounted  for,  as  would  be  the  case  if  utilities  were 
assessed  using  lotteries.  Thus,  the  resulting  assessments  of 
worth  here  are  eq\'  i.valent  to  utilities  only  if  the  commander 
is  assumed  to  be  risk-neutral.  To  insure  that  this  assump- 
tion is  not  forgotten,  we  will  refer  here  to  the  assessments 
of  worth  as  values  rather  than  utilities. 
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the  enemy  pilots  are  generally  less  qualified  than  ours,  so 
that  the  loss  of  a good  pilot,  like  the  one  which  attacked  the 
HERRINGBONE,  hurts  them  more  than  it  does  us.  They  have 
significantly  fewer  pilots  than  we  do,  but  a higher  percent- 
age of  them  are  fighter  pilots.  Of  the  fighter  pilots,  less 
than  10%  are  qualified  enough  to  fly  high-threat  or  night 
missions. 

Consultant:  Based  on  these  facts,  who  stands  to  lose 
more  from  the  loss  of  a qualified  fighter  pilot,  the  U.S.  or 
the  enemy?  And  how  much  more? 

Commander:  Things  would  be  pretty  even,  except  for  the 
extreme  shortage  of  enemy  pilots.  I would  say  that  losing  a 
qualified  fighter  pilot  hurts  them  about  three  times  as  much 
as  it  does  us.  Of  course,  this  is  just  in  terms  of  the  cost 
of  replacement,  how  much  they  are  worth  to  the  war  effort. 

Consultant:  Since  you  said  that  it  cost  about  $1 
million  to  replace  a U.S.  pilot,  this  would  mean  that  an 
enemy  pilot  is  worth  $3  million  in  "equivalent"  dollars. 
"Equivalent"  means  that  they  would  just  as  soon  lose  $3 
million  as  lose  an  aviator;  it  doesn't  mean  that  they  pay 
this  much  for  one. 

Now,  on  outcomes  D and  G,  the  U.S.  loses  three  ship's 
officers  and  22  ship's  enlisted  men.  Since  the  Navy  must 
train  these  men  as  well  as  the  aviators,  it  should  be  rela- 
tively easy  to  assess  how  much  they  are  worth. 

Commander:  Of  course,  I may  want  to  check  this  out 
more  exactly  later,  but  offhand,  I would  say  that  one  U.S. 
aviator  is  worth  two  ship's  officers,  or  five  enlisted  men. 

Consultant:  Then  the  personnel  loss  due  to  the  bombing 
of  the  ship  is  equal  to  5.9  aviators,  or  5.9  million  "equiva- 
lent" dollars.  To  help  you  visualize  all  this,  I'm  going  to 
make  a scale  of  the  values  of  the  various  outcomes  (Figure 
6-3}.  Outcomes  B,  £,  F,  and  H are  at  zero,  because  in  each 
of  these  outcomes,  no  men  are  lost.  C is  at  +$3  million, 
because  the  enemy's  loss  is  our  gain,  and  outcome  A is  at 
-$1  million,  which  is  the  cost  of  the  loss  of  our  aviator. 
Finally,  outcomes  D and  G are  at  -$5.9  million,  because  that 
is  the  loss  of  25  men  from  the  bombing  of  the  ship.  Does 
this  scale  look  reasonable  to  you?  Remember,  we're  con- 
sidering only  the  men  here,  not  equipment,  political  value, 
or  morale. 

Commander;  Yes,  that  scale  looks  about  right.  Of 
course,  since  I came  up  with  those  values  myself,  it  would 
have  to  look  right,  wouldn't  it? 
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Consultant:  Not  necessarily.  Sometimes  in  trying  to 
break  down  a complex  problem  into  its  component  parts  we 
overlook  something  Important,  so  it's  best  to  check  the 
formal  procedure  with  your  intuition  every  once  in  a while. 
When  there  is  disagreement,  this  indicates  that  something 
has  been  overlooked.  It  may  be  that  the  formal  procedure  is 
being  applied  wrongly  or  your  intuition  is  faulty  or  both. 
However,  it  is  usually  possible  to  resolve  the  difference 
without  too  much  trov&le. 

Let’s  proceed  to  the  EQUIPMENT  attribute  next.  What  is 
the  cost  of  replacement  for  the  U.S.  aircraft? 

Commander:  Well,  let's  see.  We  said  that  it  was  an 
A-4,  right?  One  of  those  costs  about  $2  million,  I would 
guess.  And  I'll  bet  that  next  you'll  want  to  know  what  the 
enemy  airplane  was  worth,  right? 

Consultant;  Right. 

Commander:  Well,  Soviet  MIG-17 's  aren't  exactly  the 
newest  and  best  airplanes  available,  not  as  if  they  were 
MIG-21 's.  I would  give  a figure  of  about  $1  million. 
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Consultant:  In  outcomes  D and  G,  the  enemy  bombs  the 
ship.  What  is  the  cost  of  that  damage? 

Commander:  Whew!  That’s  not  so  easy  to  figure  out. 

First  of  all,  there  Is  the  work  required  to  repair  the  ship, 
which  I would  guess  is  around  200  man-months.  Then,  of 
course  the  ship's  crew  is  idle  for  the  time  the  ship  is  in  dry 
dock,  so  we  have  to  figure  the  down  time  for  the  ship's 
company— make  that  300  man-months.  That  comes  to  a total  of 
500  man-months,  which  is  42  man-years;  and  figuring  about  20 
thousand  dollars  per  man-year,  that's  $840,000. 

What  else  is  there?  Oh  yes,  there's  the  cost  of  the 
materials  to  repair  the  ship,  say  $2  million,  and  of  course, 
you  have  to  bring  another  ship  on  the  line  to  replace  the 
damaged  ship.  This  means  transporting  it  across  the  ocean, 
which  costs  fuel  and  so  on.  Let's  say  this  costs  about  $1 
million.  What  does  that  come  to? 

Consultant:  The  total  is  approximately  $4  million  in 
coats,  with  this  information,  1 think  we  have  enough  to  make 
a scale  (Figure  6-4)  which  is  like  the  one  for  the  men  dimen- 
sion. The  value  of  shooting  down  the  MXG  is  +$1  million  (C) , 
the  value  of  either  shooting  at  a friend  and  missing,  or  not 
shooting  at  him  (B  or  F)  is  zero,  the  value  of  losing  an  A-4 
is  -$2  million  (A) , and  the  cost  of  the  ship  being  bombed, 
because  you  failed  to  shoot  at  an  enemy,  or  shot  at  him  and 
missed  (G  or  D)  is  -$4  million.  Oh  yes,  an  enemy  who  manages 
to  make  a bombing  run  but  misses  the  ship  (E  or  H) , that's 
worth  zero  as  far  as  equipment  costs  are  concerned.  Does  the 
scale  look  about  right?  Notice  that  the  length  of  the  whole 
scale  is  $5  million,  while  the  length  of  the  MEN  scale  is 
nearly  $9  million.  This  means  that  the  EQUIPMENT  attribute  is 
about  5/9tha  or  slightly  more  than  half  as  important  as  the 
MEN  attribute.  Do  you  agree? 

Commander:  Yes,  I would  consider  the  loss  of  life  to  be 
more  important  than  the  loss  of  equipment,  and  in  about  that 
ratio. 

Consultant:  Now,  for  the  POLITICAL  attribute.  Here, 
there  are  no  convenient  units  to  measure  things  in,  like 
dollars.  However,  I think  we  can  still  obtain  the  necessary 
values.  To  begin  with,  let's  order  the  outcomes.  What  is  the 
most  favorable  outcome  politically?  We're  talking  about  the 
reaction  back  home,  what  the  media  might  say,  and  potential 
action  by  Congress. 

Commander:  Of  course,  the  best  outcome  is  to  shoot  down 
an  enemy  airplane  (C) . 

Consultant:  What's  the  worst? 
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Figure  6-4 

MEN  AND  EQUIPMENT 


Commander:  The  wont  possible  thing  is  for  the  enemy 
to  bomb  your  ship  without  your  ever  taking  a shot  at  him 
(G) . 


Consultant:  O.K.,  now  order  the  other  outcomes  between 
C,  shooting  the  enemy,  and  G,  falling  to  shoot  at  the  enemy 
and  getting  hit. 


Commander:  Well,  the  worst  thing  next  to  G is  D,  the 
enemy  bombs  your  ship  because  you  missed  him  when  you  shot 
at  him.  Next  to  that  is  k,  shooting  down  one  of  your  own 
aircraft  and  losing  the  pilot.  After  that  is  B,  shooting  at 
one  of  your  own  men  and  missing.  I guess  that  H,  not  shooting 
at  an  :memy  whose  bomb  misses  the  ship,  is  not  quite  as  bad 
as  B,  and  a little  better  than  H is  E,  shooting  at  and 
missing  an  enemy  who  in  turn  misses  the  ship.  Finally  not 
shooting  at  one  of  your  own  aircraft,  F,  is  neutral. 


Consultant:  Now  that  you  have  ordered  the  outcomes 
(Figure  6-5),  the  next  step  is  to  judge  how  much  better  or 
worse  one  outcome  is  corap  red  to  another.  We  can  construct 
a relative  scale  of  values  using  0 for  the  worst  outcome  (G) 
and  100  for  the  best  outcome  (C) . Now,  relative  to  not 
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shooting  at  one  of  your  own  aircraft  (F) , how  much  worse  is 
it  to  have  the  ship  bombed  without  a shot  being  fired  (G), 
than  it  is  to  shoot  down  one  of  your  own  planes  (A)?  We're 
comparing  G to  A,  relative  to  F. 
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POLITICAL  VALUl' ; RANK  ORDER  Or  Ol'  l !ONii 


Commander:  I would  say  that  it  is  three  times  as  bad 
to  have  the  ship  bombed.  After  all,  we  do  occasionally 
shoot  down  one  of  our  own  planes,  and  while  it  is  not  a 
popular  thing  to  do  politically,  it  is  certainly  much  better 
than  allowing  your  ship  to  be  bombed  without  even  taking  a 
shot. 

Consultant:  Well,  since  you  said  that  F,  not  shooting 
at  one  of  your  own  aircraft,  is  neutral,  this  means  that  the 
distance  from  F to  G must  be  three  times  as  great  as  the 
distance  from  F to  A.  That  makes  it  three  times  as  bad. 

What  about  D,  shooting  at  the  enemy  but  missing  him  so  that 
he  gets  through  to  bomb  the  ship? 

Commander:  Well,  that's  almost  as  bad  as  G;  put  it 
down  very  close  to  G on  the  scale. 

Consultant:  O.K.,  what  about  shooting  at  a friend  and 
missing  him  (B)7 
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Conmander : Well,  the  pilot  or  one  of  his  buddies  might 
write  someone,  and  that  someone  could  write  his  congressman, 
and  that  would  result  in  political  pressure. 

Consultant:  How  does  it  compare  to  shooting  him  down? 

Commander:  Of  course,  shooting  him  down  is  worse,  I'd 
say  almost  twice  as  bad.  Let's  see,  that  must  mean  that  the 
distance  from  F to  B is  a little  more  than  halfway  down  the 
distance  from  F to  A. 

Consultant:  That's  good.  Now,  what  about  C,  shooting 
down  an  enemy? 

commander:  Well,  politically,  it's  just  not  that  big  a 
deal  to  shoot  down  the  enemy.  After  all.  It's  what  you're 
supposed  to  do,  so  no  one  really  pays  much  attention  when  you 
do  it.  It  certainly  doesn't  have  nearly  the  political  Impact 
of  shooting  at  one  of  your  own  airplanes,  even  when  you  mlsslj 

Consultant:  How  does  the  political  Impact  of  shooting 
down  the  enemy  (C)  compare  to  the  Impact  of  shooting  and 
missing  a friend  (B}7  Remember,  we're  making  the  comparison 
relative  to  F,  not  shooting  at  one  of  your  own  aircraft. 

Commander:  Well,  I'd  say  that  shooting  at  a friend  and 
missing  him  has  three  times  the  political  Impact. 

Consultant:  O.K.,  so  the  distance  from  F to  C is  one- 
third  the  distance  from  F to  B.  What  about  outcomes  B and  H? 
You  said  they  fall  between  F and  B;  can  you  say  where? 

Commander:  That's  difficult.  E Is  a little  worse  than 
F,  and  H Is  a little  worse  than  E but  a little  better  than  B. 
Make  the  differences  all  about  the  same. 

Consultant:  Now  we  have  enough  information  to  construct 
a scale  (Figure  6-6)  which  gives  us  the  relative  values  of 
each  of  the  outcomes  on  the  political  attribute. 

Commander:  I must  say  that  putting  numbers  to  these 

political  values  lends  spurious  precision  to  my  judgments.  I 
feel  pretty  fuzzy  about  most  of  these  values. 

Consultant:  Well,  If  we  had  fuzzy  numbers  we'd  use  them. 
But  we  don't,  so  for  the  moment,  let's  leave  the  over-precise 
numbers,  and  remember  that  each  number  represents  an  approxi- 
mate location  on  the  scale.  Later,  if  we  wish,  we  can  move 
those  nvunbers  about  to  see  the  effects  your  fuzziness  has  on 
your  decision.  That  would  be  one  kind  of  sensitivity  analysis. 
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Cononander:  I see — vre  can  Incorporate  my  vagueness 
about  these  values  by  changing  the  numbers » but  not  so  much 
that  the  ordering  of  the  outcomes  would  be  different.  I'm 
reasonably  confident  about  the  order. 

Consultants  That's  right.  Now  our  next  job  is  to 
figure  out  some  way  to  compare  these  political  values,  which 
are  on  a scale  from  zero  to  100,  with  the  MEN  and  EQUIPMENT 
values,  which  are  expressed  on  a dollar  scale.  In  other 
words,  we  need  to  get  a total  length  for  the  political 
scale,  and  this  length  represents  the  importance  of  the 
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political  dimension.  How  important  are  political  considera- 
tions! relative  to  either  men  or  equipment? 

Commander:  Well/  as  you  know,  in  this  conflict,  there 
has  been  a lot  of  public  opinion,  the  people  get  stirred  up, 
and  their  opinion  has  a lot  more  weight  than  it  would  have  had 
back  in  World  War  II.  I would  say  that  in  this  situation,  I 
consider  political  value  to  be  about  twice  as  important  as 
men. 


Consultant:  So,  we  will  now  stretch  the  political  value 
scale  out  to  two  times  the  length  of  the  scale  for  men.  Since 
the  total  length  of  the  scale  for  men  Is  about  $9  million,  the 
total  length  of  the  political  scale  must  be  18,  and  we  can 
call  these  18  million  "equivalent  dollars."  This  makas  the 
political  scale  between  three  and  four  times  as  long  as  the 
equipment  scale,  which  is,  recall,  $5  million  long.  Does  that 
seem  about  right? 

Commander:  Yes,  political  value  is  considerably  more 
important  than  the  value  of  equipment. 

Consultant:  Now  let's  determine  the  equivalent  dollar 
values  of  the  outcomes  on  the  political  scale.  Outcome  F is 
neutral,  so  we  will  place  it  at  zero.  Now,  to  keep  all  the 
relative  distances  about  the  same,  as  in  Figure  6-6,  I need  to 
make  the  distance  from  F to  A one-third  the  distance  from  F to 
G.  A little  over  half  the  way  from  F to  A,  I'll  put  B.  Then, 
I'll  go  up  from  F about  one-third  of  the  distance  from  F to  B 
to  locate  C.  Let's  put  C at  1.0.  If  I put  G at  -17,  then  A 
will  be  a little  less  than  -5,  and  B will  be  about  -3.  You 
said  D was  next  to  G,  where  would  you  put  it? 

Commander:  Put  it  at  -16. 

Consultant:  That  completes  the  scale  (Figure  6-7) . Now, 
does  it  agree  with  your  intuition?  Check  both  the  location  of 
the  outcomes  on  the  scale,  and  also  check  the  scale  against 
the  other  scales. 

Commander:  I think  this  is  a pretty  good  picture  of  the 
situation.  It  looks  as  if  the  political  value  of  shooting 
down  the  wUemy  is  the  same  as  the  equipment  value. 

Consultant:  That's  right.  Although  the  political  value 
of  C is  small  compared  to  other  outcomes,  the  political 
attribute  is  so  important  that  it  makes  up  for  it.  Notice 
that  we  have  done  an  interesting  thing.  We  have  taken  a 
completely  subjective  scale,  political  value,  and  related  it 
to  a completely  objective  scale,  dollars.  Thus,  we  can  talk 
about  political  value  in  terms  of  millions  of  equivalent 
dollars*  and  when  we  combine  the  different  attributes  to  get 
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an  overall 
of  millions 


(At  this  point,  the  consultant  and  the  Force  Commander 
nt  through  the  morale  attribute  in  a similar  fashion. 

First,  they  ordered  the  outcomes,  bhen  they  scaled  them,  and 
finally,  they  weighted  the  entire  attribute.  As  with  the 
other  three  attributes,  outcome  F was  judged  to  have  no 
positive  or  negative  morale  value,  and  thus,  defined  the 
zero  point.) 

Consultant:  This  is  the  final  set  of  scales  (Figure  6-8) . 
The  lengths  correspond  to  their  importances,  and  the 
relative  position  of  each  of  the  outcomes  on  any  attribute 
indicates  its  relative  value  on  that  attribute.  Does  every- 
thing look  O.K.? 

Conmiander:  Yes,  I think  those  are  the  values  1 would 
use  in  making  a decision  in  this  situation.  At  least,  1 
can't  see  anything  which  disagrees  with  my  intuitions. 

Consultant:  Now  that  we  have  assessed  your  values  on 
each  attribute,  we  know  what  each  of  the  outcomes  would  be 
worth  to  you  if  it  occurred.  The  value  of  each  outcome  is 
simply  the  sum  of  the  different  values  of  men,  equipment, 
political  value,  and  morale  which  go  with  each  one.  This 
means  that  A is  worth  about  -10  million  equivalent  dollars, 

B is  worth  -3.3  million,  C is  worth  +5.8,  and  so  on,  as 
shown  on  our  decision  diagram  (Figure  6-9) . 

Branch  Probabilities 

Consultant:  We  still  don't  know  what  decision  to  make, 
because  we  don't  know  which  of  the  possible  outcomes  will  be 
the  one  that  occurs.  In  order  to  compare  the  two  options, 

SHOOT  and  NO  SHOOT,  we  need  to  put  probabilities  on  each  of 
the  branches  coming  from  an  event  fork)  that  is,  we  must 
assess  your  uncertainty  about  which  of  the  branches  will 
occur. 


Working  from  the  right  to  the  left,  the  first  event 
fork  is  the  one  which  determines  whether  you  hit  the  enemy 
or  friendly  aircraft  when  you  shoot  at  it.  How  likely  is  it 
that  the  gun  crews  can  hit  an  aircraft  in  this  situation 
when 'they  shoot  ut  it?  Remember  that  we  have  been  assuming 
that  a "hit"  means  that  they  destroy  the  aircraft  and  kill 
the  pilot. 

Comraander:  Well,  let's  see.  They  can't  be  absolutely 
certain  of  hitting  it,  nor  is  it  impossi.ble.  The  chances 
are  definitely  better  than  50*50.  I'd  say  that  the  proba- 
bility is  about  0.70  that  tht  crews  will  hit  the  aircraft  if 
they  shoot  at  it. 

Consultant:  O.K.,  we'll  put  a probability  of  0.70  on 
the  "kill"  branch,  and  since  they  either  hit  the  plane  or 
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they  don't,  we'll  put  a 0.30  on  the  "don't  kill"  branch. 
Does  it  make  any  difference  in  the  probability  of  hitting 
the  aircraft  whether  it  is  friendly  or  an  enemy? 

Coirminders  I remember  you  asked  me  earlier  if  the 
chance  of  a hit  is  affected  by  the  type  of  plane,  but  on 
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Figure  6-9 


reflection,  I'd  have  to  say  no.  We  would  use  a missile,  and 
with  this  angle  of  attack,  the  probability  stays  the  same 
whether  it's  a friend  or  an  enemy. 

Consultant:  Then  I'll  put  0.70  and  0.30  on  both  sets 
o^  branches.  What  about  the  probability  that  the  aircraft 
is  a friend  or  an  enemy? 

Commander:  Well,  the  ship  was  conducting  shore  bombard- 
ment, and  about  80%  of  all  this  type  of  bombardment  takes 
place  south  of  Point  Bunker.  If  the  HERRINGBOKE  had  been  in 
that  area,  1 would  have  said  that  there  was  only  about  a 1% 
chance- of  an  enemy  attack,  mostly  because  there  has  never 
been  an  enemy  attack  before.  It  is  true,  however,  that  the 
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enemy  had  been  flying  some  risky  missions  at  this  time, 
trying  to  attack  U.S.  bombers.  About  18%  of  the  shore 
bombardments  take  place  right  around  Point  Bunker,  and  the 
remaining  2%  occur  significantly  farther  north,  where  the 
HERRINGBOKE  was.  In  this  area,  I would  have  to  give  a 
probability  of  about  15%  of  an  enemy  attack. 

Consultant:  That's  the  szune  as  about  one  chance  in 
six,  and  leaves  five  chances  in  six  that  the  aircraft  is 
friendly.  Do  you  agree? 

Commander:  Yes,  that's  about  right. 

Consultant:  Now  look  at  the  bottom  right  fork.  What 
is  the  chance  that  the  enemy  plane  will  score  a hit  on  the 
ship  when  you  haven't  shot  at  him? 

Commander:  1 could  give  you  a fairly  informed  assess- 
ment after  consulting  some  tables  giving  hit  probabilities 
for  different  types  of  targets,  weapons,  and  so  forth.  But 
offhand,  I'd  say  that  for  a target  like  the  HERRINGBONE  and 
the  type  of  approach  that  would  have  to  be  taken  by  the  pilot 
of  the  MIG-17— well,  all  things  considered,  about  35%. 

Consultant:  Does  that  probability  change  if  you  are 
shooting  at  him? 

Commander:  Oh  yes.  He's  under  pressure  and  will  find 
it  more  difficult  to  make  a clean  run  on  the  ship.  He's 
also  more  likely  to  make  a mistake.  I'd  say  he  has  a 20% 
chance  of  hitting  the  ship. 

Consultant:  So,  we  put  35-65  on  the  branches  of  the 
lower  right  fork,  but  we  assign  20-80  to  the  branches  of  the 
HIT  - NO  HIT  fork  just  above.  We  now  have  all  the  values  for 
all  of  the  possible  outcomes,  and  the  branch  probabilities 
for  all  of  the  event  forks.  Our  decision  diagram  (Figure 
6-10)  now  contains  enough  information  to  reach  the  best 
decision. 

Maximization  of  Expected  Value 

Consultant:  To  reach  the  best  decision,  we  apply  this 
decision  rule:  Choose  that  act  which  has  associated  with  it 
the  highest  expected  value.  You  might  think  of  an  expected 
value  as  a sort  of  figure  of  merit,  just  as  the  values  at 
the  end  of  the  tree  indicate  the  relative  worth  or  merit  of 
the  outcomes  A through  H.  The  outcomes  range  in  value  from 
-$28.8  million  for  G to  +$5.8  million  for  C.  Every  outcome 
falls  on  that  scale  from  -$28.8  to  +$5.8.  Now,  by  applying 
the  expected  value  principle,  we  can  put  the  acts  on  that 
same  scale,  thereby  enabling  us  to  compare  easily  the  rela- 
tive desirability  of  the  acts.  To  find  the  expected  value 
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Figure  6-10 


of  the  acts,  we  start  at  the  right  of  the  tree  with  the 
values  for  the  outcomes  and  work  back  to  the  acta. 

To  see  how  this  works,  let's  start  with  the  bottom 
right  event  fork,  HIT  or  NO  HIT  following  not  shooting  at 
the  enemy.  What  is  the  value  of  not  shooting  at  an  enemy? 

Commander:  It's  either  -28.8  or  -2.5  depending  on 

whether  he  successfully  bombs  the  ship. 
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consultant:  Yes,  the  value  of  not  shooting  an  enemy  if  | 

he  hits  you  is  -$28.8,  or  ^ he  doesn't  hit  you,  -$2.5.  But 
what  is  the  value  just  of  not  shooting  at  an  enemy? 

Commander:  Well,  since  I'm  not  sure  that  he'll  hit  the 
ship,  the  value  you  want  must  be  something  less  than  -$28.8. 

And,  by  the  same  line  of  reasoning,  since  I'm  not  sure  he 
will  miss  the  ship,  the  value  must  be  worse  than  -$2.5. 

Something  between  -$28.8  and  -$2.5.  But  how  much? 

Consultant:  That's  where  the  probabilities  come  in. 

If  you  thought  the  chances  of  HIT  or  NO  HIT  were  50-50,  then 
not  shooting  at  an  enemy  could  be  valued  as  half  way  between 
-28.8  and  -2.5,  or  -13.15.  Here,  the  probabilities  are  35- 
65,  so  we  assign  to  NO  SHCOT  a value  that  is  65%  of  the 
distance  from  -28.8  to  -2.5.  That's  -$11.7.  So,  on  our 
value  scale  for  the  outcomes,  which  extends  from  -28.8  to 
4-5.8,  we  have  placed  not  shooting  at  an  enemy  at  -11.7. 

Commander:  Aren't  you,  in  effect,  using  the  proba- 
bilities as  weighting  factors?  The  more  sure  you  are  of  a 
miss,  the  more  heavily  you  weight  that  -$2.5  outcome? 

Consultant:  Yes,  in  fact,  that's  what  an  expected 
value  is:  a weighted  average.  To  calculate  an  expected 
value,  you  multiply  each  value  by  its  probability  and  add  up 
over  all  the  branches  of  the  fork.  For  the  bottom  fork: 

Value  X Probability 

-28.8  X .35  » -10.1 

- 2.5  X .65  “ -1.6 

SUM  » 11.7  ■ Expected  value. 

We  write  that  expected  value  in  the  circle  at  the  node  of 
the  branch  (Figure  6-11). 

Commander:  I see  now  how  we  arrive  at  an  expected 
value  for  not  shooting  at  an  enemy,  but  how  do  we  find  the 
expected  value  of  the  act  NO  SHOOT? 

Consultant:  We  apply  the  same  approach  again.  Not 
shooting  at  a friend  is  valued  at  0,  and  we've  just  found 
that  not  shooting  at  an  enemy  is  valued  at  -$11.7.  To  find  j 

the  expected  value  of  not  shooting,  we  weight  those  two 
values  with  their  respective  probabilities,  0.85  and  0.15, 
and  add: 


Value  X 


Probability  j 


0 X 
-11.7  X 


0.85  » 0 

0.35  « -1.8 

SUM  » . 
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Figure  6-11 


The  expected  value  of  not  shooting  Is  and  we  write  It 

in  the  circle  following  NO  SHOOT.  Now  we  know  where  the  act 
NO  SHOOT  stands  on  the  value  scale  from  -28.8  to  -4-5.8. 

Commander:  I see  how  you  get  the  expected  value/  but 
what  does  it  mean?  It  doesn't  have  any  reality,  does  it? 

Consultant:  Remember  that  the  -$1.8  is  not  the  value 
of  any  r'lal  outcome;  it  1 s the  value  of  not  shooting,  taking 
into  account  the  uncertainty  about  whether  he  is  a friend  or 
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an  enemy,  and  the  uncertainty  about  whether  an  enemy  would 
score  a hit  on  the  ship.  The  number  is  an  index  of  the 
value  of  not  shooting,  relative  to  the  best  and  worst  out- 
comes valued  at  +5.8  and  -28. B. 

Commander:  I see.  Now,.  I suppose,  we  apply  this  same 
weighting  process  to  get  the  expected  value  of  shooting. 

Consultant:  That's  right.  You  might  start  with  the 
upper  right  fork.  What  is  the  value  of  shooting  at  a friend? 

Commander:  It's  the  weighted  average  of  -10  and  -3.3: 

-10  X 0.7  » -7.0 

-3.3  X 0.3  = -0.99 

SUM  » -7.”d5,  say  -8.0. 

Shooting  at  a friend  is  valued  at  -$8.0,  right? 

Consultant:  Yes.  Now  what  about  shooting  at  an  enemy? 

Commander:  That  should  be  a weighted  average  of  the 
outcomes  from  KILL  and  NO  KILL.  But  the  latter  involves 
another  event  fork,  HIT  and  MO  HIT,  so  I guess  we  should 
start  there. 

Consultant:  Yes,  the  value  of  shooting  at  an  enemy  and 
missing  him  gives  an  outcome  of  -$2€.9  if  his  bomb  hits  the 
ship,  or  -$1.2  otherwise.  Weighting  these  outcome  values 
with  the  probabilities  0.2  and  0.8  gives  an  expected  value 
of  (-26.9)  X (0.2)  + (-1.2) (0.8)  = -6,3.  Shooting  at  an 
enemy  and  missing  him  is  valued  at  -$6.3.  Why  don't  you 
work  out  the  value  of  shooting  at  an  enemy? 

Commander:  It's  70%  of  +$5.8  plus  30%  of  -$6.3.  I get 
$2.2.  That's  the  first  positive  expected  value  we  ha/e 
obtained.  Does  that  mean  that  there  is  a positive  benefit 
in  shooting  at  an  enemy? 

Consultant:  Relative  to  not  shooting  at  a friend, 
which  is  valued  at  0,  yes.  But  you  could  have  obtained  a 
negative  expected  value  there  if  the  chance  of  NO  KILL  had 
been  high,  for  then  the  -6.3  would  have  been  weighted  more 
heavily,  or  if  the  chance  of  HIT  had  been  higher,  for  then 
the  -26.9  would  have  counted  more. 

Commander:  Are  you  saying  that  I can't  just  choose  a 
course  of  action  on  the  basis  of  whether  or  not  it  has  a 
positive  or  negative  expected  value  associated  with  it? 

Consultant:  Right.  The  decision  rule  is  to  choose  the 
act  with  the  highest  expected  value.  In  some  situations, 
all  courses  of  action  will  have  negative  expected  values: 
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then  you  choose  the  one  that's  least  bad.  Sometimes  they 
may  be  all  positive;  in  that  happy  circumstance,  you  choose 
the  act  with  the  highest  positive  expected  value.  Now  let's 
finish  the  calculations.  What  is  the  expected  value  for 
shooting? 

Commander:  Well,  shooting  at  a friend  is  valued  at  '-$8.0, 
while  shooting  at  an  enemy  is  worth  +$2.2.  Weighting 
those  values  with  the  probabilities  of  FRIEND  and  of  ENEMY 
gives  (-8.0)  x (0.85)  + (+2.2)  x (0.15)  - -6.5.  So,  shooting 
is  valued  at  -$€.5. 

Consultant:  Right.  Now,  If  we  show  on  the  decision 
diagram  (Figure  6-11)  all  the  expected  values  we  have  calcu- 
lated, we  are  clear  to  the  left  fork.  What  kind  of  fork  is 
it? 


Commander:  It's  an  act  fork. 

Consultant:  Which  means  that  the  way  things  go  is 
under  your  control.  You  get  to  choose  which  branch  to  take. 
Which  one  will  you  take? 

Commander:  One  branch  is  valued  at  -6.5,  and  the  other 
one  at  -1.8.  It  looks  like  I lose,  no  matter  which  one  I 
choose. 

Consultant:  Well,  that's  typical  of  this  kind  of 
tactical  defense  situation  in  which  a relatively  inexpensive 
enemy  airplane  can  do  a lot  of  damage  to  a relatively  expen- 
sive ship.  You're  in  a tough  situation:  if  you  shoot  and 
it's  a friend,  then  you've  made  a mistake.  On  the  other 
hand,  if  you  don't  shoot,  and  it  turns  out  to  be  an  enemy, 
you'vf^  made  another  kind  of  mistake.  Still,  you  must  choose. 

Commander:  Since  the  NO  SHOOT  branch  is  -1.8,  it's 
better  than  the  SHOOT  branch,  which  is  -6.5.  I choose  to 
hold  my  fire. 

Consultant:  Our  decision  rule  says  that  it's  the  best 
thing  to  do  in  this  situation.  If  the  captain  of  the 
HERRINGBONE  had  values  and  probabilities  like  those  in  the 
decisio.n  diagram,  then  he  did  the  correct  thing,  even  though 
the  aircraft  turned  out  to  be  an  enemy  and  bombed  him. 
Decision  theory  doesn't  eliminate  mistakes;  it  merely  helps 
you  avoid  those  which  are  avoidable. 

Notice  one  thing:  the  probability  that  the  aircraft  is 
friendly  or  hostile  is  very  important  to  the  decision. 

After  all,  if  you  knew  for  certain  that  it  was  an  enemy, 
then  the  expected  values  at  the  act  fork  would  be  +2.2, 
and  -11.7,  and  you  would  shoot.  If,  on  the  other  hand,  you 
knew  for  certain  that  it  was  a friend,  the  expected  values 
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would  be  -8.0  and  0,  you  wouldn't  shoot.  Using  these  values 
as  extremes,  we  can  draw  a graph  (Figure  6-12)  which  shows 
how  the  expected  value  of  shooting  and  not  shooting  changes 
as  the  probability  that  it's  an  enemy  Increases  from  0.0  to 
1.00.  Where  the  two  lines  cross  is  where  you  should  change 
your  decision.  In  this  case,  you  shouldn't  shoot  if  the 
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probability  of  enemy  iti  less  than  37%.  But  if  the  proba- 
bility is  greater  than  37%,  you  should  shoot.  The  decision 
analysis  has  enabled  us  to  derive  a useful  rule  of  thumb 
that  can  be  applied  quickly  if  a similar  situation  arises. 

Sensitivity  Analysis 

Commander:  One  of  the  problems  I'm  concerned  with  is 
the  spurious  precision  of  the  numbers  l have  been  giving. 

In  many  cases  they  are  my  personal  judgments,  and  I am 
rather  fuzzy  about  some  of  the  assessments.  If  I use  this 
type  of  analysis  to  make  tactical  decisions  or  to  make 
recommendations  about  information  systems  or  weapons  systems, 
I want  to  know  if  my  decisions  or  recommendations  would  be 
different  if  I had  given  different  numbers  for  some  of  my 
fuzzier  assessments.  What  should  I do? 

Consultant:  The  very  first  thing  that  you  should  do  is 
a sensitivity  analysis.  This  kind  of  analysis  will  tell  you 
how  important  the  fuzz  in  your  assessments  is.  If  a par- 
ticular number  can  vary  quite  a bit  without  affecting  the 
result,  then  the  fuzz  doesn't  matter.  On  the  other  hand,  if 
a small  difference  in  one  of  the  values  makes  a large  dif- 
ference in  the  result,  it  is  important  that  you  take  time  to 
make  the  best  possible  assi^ssment.  Which  of  the  input 
values  that  we  have  used  in  this  problem  are  you  roost  unsure 
of? 

Commander:  Well,  one  of  the  numbers  which  seems  to  be 
very  important  in  the  analysis  is  the  probability  of  kill 
for  the  current  weapons  system.  I said  there  was  a 70% 
probability,  but  I didn't  get  that  figure  by  counting  the 
number  of  times  that  it  hit  a target  under  these  conditions, 

I merely  gave  my  best  judgment. 

Consultant:  Well,  that  may  be  good  enough.  How  sure 
are  you  that  the  weapons  system  has  a kill  probability 
greater  than  50%;  that  is,  is  it  more  likely  to  hit  the 
aircraft  than  not  hit  it? 

Commander:  Oh,  I am  very  sure  of  that. 

Consultant;  On  the  other  hand,  we  know  that  the  proba- 
bility of  kill  cannot  be  greater  than  100%.  In  order  to 
show  the  sensitivity  of  our  decision  to  shoot  or  not  to 
shoot,  let  us  take  our  original  graph  (Figure  6-12)  which 
showed  the  expected  value  of  each  decision  as  a function  of 
the  probability  of  an  enemy  and  try  changing  the  probability 
of  kill.  Of  course,  if  we  decide  not  to  shoot,  the  proba- 
bility of  kill  is  irrelevant  so  the  NO  SHOOT  line  will  stay 
the  san:e.  However,  the  expected  value  of  shooting  certainly 
depends  on  the  probability  of  kill.  So  let's  work  out  a new 
SHOOT  line  by  assuming  100%  probability  of  kill,  then 
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another  line  by  assuming  the  lowest  possible  probability  of 
kill,  50%.  Take  a look  at  the  decision  diagraun  (Figure  6- 
11).  First,  we  assume  that  the  probability  of  kill  is  100%. 

If  we  decide  to  shoot  and  the  aircraft  turns  out  to  be  an 
enemy,  what  is  the  expected  value? 

Commander:  Well,  with  100%  chance  of  kill,  the  ex- 
pected val'^e  will  be  +5.8. 

I 

Consultant:  Right.  What  if  we  shoot  and  it  turns  out 
to  be  a friend? 

Commander:  Killing  a friend  was  valued  at  -10.0. 

Consultant:  That's  right.  Let's  put  thosa  two  points 
on  a graph  and  draw  a straight  line  between  them  (Figure  6- 
13).  This  line  represents  the  expected  va'^ue  for  shooting, 
when  you  have  100%  probability  of  kill,  as  a function  of  a 
probability  that  it  is  an  enemy.  Now,  since  you  are  sure 
that  the  probability  of  kill  is  at  least  50%,  let's  use  that 
as  the  other  extreme,  if  there  is  50%  kill  probability,  you 
shoot  and  it  turns  out  to  be  a friend,  then  the  expected 
value  must  be  halfway  between  the  value  for  100%  probability 
of  kill  and  the  value  for  zero  probability  of  kill  from 
Figure  6-13.  Halfway  between  -10.0  and  -3.3  is  -6.7.  Again 
referring  to  Figure  6-11,  if  you  shoot  when  you  know  for 
sure  it  is  an  enemy  and  you  have  a 50%  kill  probability, 
then  the  expected  value  would  be  halfway  between  -6.3  and  +5.8, 
or  -0.3.  Now  if  we  draw  a line  connecting  -6.7  and  -0.3  on 
our  graph,  it  will  represent  the  expected  value  of  shooting, 
when  there  is  a 50%  probability  of  kill,  as  a function  of 
the  probability  of  an  enemy  (Figure  6-13) . The  important 
thing  to  notice  on  this  graph  is  where  each  of  the  SHOOT 
lines  crosses  the  NO  SHOOT  line.  That  is,  at  what  threshold 
probability  of  an  enemy  will  you  change  your  decision  from 
DON'T  SHOOT  to  SHOOT.  As  we  have  already  seen,  at  70% 
probability  of  kill  the  change  occurs  when  the  probability 
of  an  enemy  reaches  37%.  Increasing  the  probability  of  kill 
to  100%  or  reducing  it  to  50%  has  hardly  any  effect  on  the 
threshold  probability — it  stays  at  37%,  give  or  take  1%. 

Commander:  So  even  if  the  kill  probability  of  my 
weapons  systems  varies  between  hitting  the  target  half  the 
time  and  hitting  it  all  the  time,  the  effect  on  my  decision 
is  negligible. 

Consultant;  That's  right,  and  therefore  it  is  not  too 
important  that  you  he  able  to  assess  the  probability  of  kill 
for  your  current  weapon  system  to  a high  degree  of  accuracy, 
at  least  for  this  problem.  Now  let's  try  another  sensitivity 
analysis.  Which  of  the  value  dimensions  are  you  most  unsure 
of? 
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Conunander:  The  thing  that  bothers  n\e  moat  about  the 
value  dimensiona  is  the  relative  importance  of  the  political 
factor.  Until  an  incident  of  this  kind  happens » it  is 
reully  difficult  for  me  to  say  how  important  political 
factors  should  be  in  my  decision. 

Consultant;  In  the  original  diagram  you  gave  the 
political  value  attribute  twice  the  importance  of  the  MEW 
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attribute.  What  would  be  the  effect  if  you  assessed  politi- 
cal value  to  be  half  the  importance  of  the  value  of  men?  In 
order  to  find  outr  we  go  back  to  the  original  value  .issess- 
ments  (Figure  6-8)  and  divide  all  the  numbers  on  the  politi- 
cal value  scale  by  4.  Then  we  add  up  all  the  values  as 
before  and  put  them  on  the  decision  diagram.  Assuming  the 
probability  of  kill  stays  at  70%/  we  can  now  generate  new 
SHOOT/NO  SHOOT  lines  corresponding  to  the  changed  values.^ 
When  I do  this,  I come  up  with  the  two  dotted  lines  shown  on 
Figure  6-14.  The  solid  lines  are  the  results  of  the  original 
analysis.  As  you  can  see/  the  effect  of  reducing  the 
political  value  i s to  raise  the  expected  value  of  any  of  the 
outcomes.  This  just  means  that  there  is  less  risk.  How- 
ever/ notice  what  happens  to  the  intersection  of  the  two 
lines / the  point  at  which  one  decides  to  start  shooting. 

The  threshold  probability  has  changed  from  .37  to  .34/  a 
small  difference.  The  reason  for  this  is  that  the  outcomes 
are  located  at  approximately  the  same  relative  positions  on 
each  of  the  attribute  scales.  Outcome  C is  always  at  the 
top,  followed  by  B,  E,  F and  H all  together  or  relatively 
close/  then  A,  with  D and  G at  the  bottom.  When  this  occurs, 
the  scales  are  highly  cc'-related.  Therefore,  changing  the 
relative  importance  can  change  the  expected  values  of  all 
the  outcomes  but  will  have  in  general  a very  small  effect  on 
the  docision.  This  insensitivity  to  value  implies  two 
things.  First  of  all,  it  is  not  too  critical  that  you  be 
able  to  assess  accurately  the  relative  importance  of  the 
political  dimension;  and  second,  changing  the  importance 
weight  for  the  political  dimension  (for  example,  if  this 
situation  were  to  occur  in  a different  political  climate) 
will  not  have  much  of  an  effect  on  the  decision. 

Impact  of  Information 

Commander:  Since  the  probability  that  the  aircraft  is 
a friend  or  an  enemy  is  so  important,  is  there  a way  to 
assess  the  impact  additional  information  might  have  on  the 
decision?  In  a situation  like  the  one  we  have  analyzed, 
there  might  be  information  available  which,  if  utilized, 
could  affect  the  decision. 

Consultant;  Yes,  there  is  a way,  and  you  are  quite 
right  in  asking  the  question.  Information  which  would 
change  the  probabilities,  making  you  more  certain,  should  be 
worth  a great  deal  to  you.  If  information,  no  matter  how 
much  or  how  precise,  won't  change  your  decision,  then  it  is 


^The  reader  is  encouraged  at  this  point  to  put  the  values 
obtained  into  the  decision  diagreim,  fold  back  the  diagram 
and  construct  the  expected  value  lines  for  the  SHOOT/ NO 
SHOOT  decision  as  before. 
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of  no  value  to  you.  Let's  discuss  first  how  new  information 
might  change  your  prior  opinion  (your  opinion  prior  to 
obtaining  the  information)  about  the  probability  of  friend 
or  enemy . 
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Bayes'  Theorem;  The  Revision  of  Opinion 

Consultant:  The  best  known  method  for  determining  the 
impact  of  information  on  your  opinion  is  called  Bayes' 
Theorem: 


P(H^|D)  P(D1H3^)  P(H^) 

P(H2|D)  ” noiiqr  * P(H2) 

where  means  that  hypothesis  1 is  true,  H2  means  that 
hypothesis  2 is  true,  and  D represents  the  data  you  have 
observed.  The  theorem  says  that  the  odds  in  tavor  of  one 
hypothesis  over  the  other,  after  you  observe  the  data,  are 
equal  to  the  odds  in  favor  of  the  hypothesis  before  you 
observed  the  data,  multiplied  by  the  likelihood  ratio.  The 
likelihood  ratio  indicates  how  much  more  likely  it  would  be 
to  observe  the  data  if  one  hypothesis  were  true,  than  if  the 
other  were  true.  So,  if  your  odds  in  favor  of  one  hypothesis 
were  3:1,  that  is,  it  was  three  times  as  likely  that  hypothe- 
sis 1 was  true  than  hypothesis  2,  and  you  saw  some  datum 
which  was  twice  as  likely  to  have  been  observed  if  hypothesis 
2 were  true  than  if  hypothesis  1 were  true,  then  your  odds 
in  favor  of  hypothesis  1 after  you  observed  the  datum  are; 

P(H, |d)  3 1 3 
■ w — X — ■ — 

P(H2|D)  1 2 2 . 

The  original  odds  of  3 to  1 correspond  to  probabilities  of 
0.75  and  0.25  (the  probabilities  are  in  a ratio  of  3 to  1) , 
and  the  revised  odds  of  3 to  2 can  be  expressed  as  probabili- 
ties of  0.60  and  0.40  (they  are  in  a ratio  of  3 to  2)  Thus, 
your  probability  in  favor  of  hypothesis  1 has  decreased  from 
0.75  to  0.60  because  you  have  observed  some  datum  which  favors 
the  other  hypothesir; — though  not  so  much  as  to  reverse  your 
original  belief  that  Hi  is  more  likely  to  be  true  than  H2. 

Now,  let's  hypothesize  possible  information  Inputs  to 
the  captain  of  the  USS  HERRINGBONE,  the  probabilities  asso- 
ciated with  them,  and  apply  Bayes'  Theorem  to  see  how  they 
might  affect  our  decision. 

What  is  the  first  information  input  that  comes  to  mind 
in  this  situation? 

Commander:  There  might  have  been  a tip-off  that  MiG's 
from  a nearby  airfield  were  in  the  air. 

Consultant:  O.K.,  there  are  two  ways  we  can  evaluate 
the  effect  of  this  information  input  on  your  uncertainty 
about  whether  the  aircraft  is  a friend  or  an  enemy.  One 
approach  is  for  you  to  revise  your  original  assessment  of 
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0.85-0.15  to  take  account  of  the  new  information,  the  tip- 
off.  If  we  do  this,  there  isn't  any  need  to  apply  Bayes' 
Theorem,  for  you  are  assessing  the  impact  of  the  information 
intuitively.  Another  approach  is  to  assess  likelihood 
ratios,  and  then  apply  Bayes*  Theorem  to  combine  the  like- 
lihood ratios  with  your  original  assessment  to  give  the 
desired  revised  assessments.  In  the  first  approach,  I will 
ask  you  to  tell  me  how  much  more  likely  it  is  that  the 
aircraft  is  a friend  or  an  enemy  assuming  that  we've  re- 
ceived a tip-off.  In  the  second  approach,  I will  ask  you 
how  much  more  likely  it  is  to  receive  a tip-off  if  the 
aircraft  is  a friend  than  if  it  is  an  enemy.  Sometimes  one 
way  is  easier  than  the  other  to  think  about  in  the  situation 
Which  is  easier  for  you? 


Commander:  Well,  it's  easier  for  me  to  imagine  that 
it's  a friend  or  an  enemy  and  then  assess  the  likelihood  of 
receiving  a tip-off.  If  it's  a friend,  1 would  say  that  you 
would  get  a tip-off  only  about  5%  of  the  time.  In  the  case 
of  an  enemy,  thr  percentage  goes  up  to  about  20%. 

Consultant:  Well,  assuming  you  received  a tip-off,  we 
can  calc" late  the  effect  of  this  information  by  using  the 
likelj.hoods  in  Bayes'  Theorem; 


p(e1t)  p(t1e)  P(E) 

B X 

p(f|t)  P(T  F)  P(F) 


.20  .15  .03  .41 
.05  .85  .0425  .59 


Receiving  a tip-off  that  MiG's  were  in  the  air  changed 
your  opinion  from  a 15%  chance  to  a 41%  chance  that  it  was 
an  enemy  aircraft.  Now,  one  nice  property  of  Bayes'  Theorem 
is  that  it  allows  you  to  use  the  posterior  probabilities 
from  the  first  piece  of  information  as  the  prior  probabili- 
ties for  the  next  piece  of  information.  This  makes  sense, 
since  youi  opinion  after  the  first  information  is  your 
opinion  before  the  second.  What  is  the  next  information 
input  likely  in  this  situation? 

Commander;  The  operator  of  the  passive  ECM  would 
ordinarily  pick  up  the  enemy  search  radar  next,  if  it  were 
an  enemy  and  if  he  had  his  radar  switched  on. 

Consultant;  What  are  the  likelihoods? 

Commander:  Well,  neither  friendly  nor  enemy  aircraft 
are  very  likely  to  be  picked  up  by  the  ECM  operator.  First 
of  all,  it  is  extremely  unlikely  that  any  friendly  airplanes 
have  radar  parameters  which  are  sufficiently  like  the 
enemy's  to  be  detected  by  the  ECM  operator.  It's  also 
extremely  unlikely  that  the  enemy  would  leave  their  radar  on 
in  this  situation  when  they  attack.  It's  kind  of  hard  to 
say  how  much  more  unlikely  one  is  than  the  other. 
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Consultant;  All  right.  What  if  you  don't  get  an  ECM 
report?  Is  that  more  likely  from  a friend  or  an  enemy? 

Commander;  It's  about  the  seune.  A friend  is  just  as 
likely  as  an  enemy  to  generate  no  ECM  report. 

Consultant;  O.K.  Let's  assume  you  don't  get  an  ECM 
report.  This  means  your  posterior  probabilities  are  the 
same  as  your  prior  probabilities; 
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Mow,  what  other  information  might  be  available  to  you? 

Commander;  The  next  thing  would  probably  be  IFF. 

(Note;  IFF  is  a device  in  friendly  aircraft,  which,  when 
turned  on,  returns  a coded  identifying  signal  in  response  to 
an  interrogating  signal.)  Now,  ordinarily,  you  would  think 
that  IFF  would  be  a good  indicator  because  all  of  our  air- 
planes have  it,  and  it's  coded  to  prevent  the  enemy  from 
imitating  it.  But,  in  a situation  such  as  this,  if  it  is  a 
friendly  airplane,  he  may  be  ditching,  and  therefore,  he  may 
have  his  IFF  gear  shot  up  or  it  might  be  turned  off.  The 
pilot  would  have  turned  it  off  before  his  bombing  run,  and 
he  might  have  forgotten  to  turn  it  back  on  as  he  came  back 
to  the  ship. 

Consultant:  What  are  the  likelihoods? 

Commander:  If  the  airplane  is  friendly,  I'd  say  there 
is  about  a 75%  chance  of  getting  an  IFF  signal  back.  If 
it's  an  enemy,  there  is  about  a 99%  chance  of  no  signal. 

The  1%  is  to  take  care  of  the  fact  that  they  may  have  dupli- 
cated the  code  and  might  mimic  the  signal. 

Consultant;  Let's  assume  that  no  IFF  return  signal  has 
been  received.  Applying  Bayes'  Theorem  again,  we  get; 

P(e1t,"ECM,-IFF)  .99  .41  .4059  .73 

P(f1t,~ECM,-IFF)  .25  .59  .1475  .27  . 

So,  on  the  basis  of  getting  a tip-off,  not  picking  up  enemy 
radar,  and  not  getting  an  IFF  signal,  the  probability  of  an 
enemy  has  moved  up  from  15%  to  73%.  Is  there  any  additional 
information  which  might  be  available? 

Commander:  Yes,  there  might  be  a visual  report.  The 
members  of  the  crew  might  confirm  visually  that  it's  an 
enemy  airplane,  or  one  of  ours,  or  there  might  be  no  report  at 
all. 
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Consultant:  So  really  there  are  three  possibilities; 

visual  report  of  enemy,  visual  report  of  friend,  and  no 
report.  What  are  the  likelihoods? 

Commander;  Let  me  put  them  down  on  paper. 

The  Commander's  final  assessments  looked  like  this: 


FRIEND  ENEMY 


VISUAL  (F) 


NO  REPORT 


VISUAL  (E) 


0.80 

0.02 

0.19 

0.48 

0.01 

0.60 

Consultants  Assuming  you  received  no  report,  and 
applying  Bayes'  Theorem,  we  haves 

P(e1t,~ECM,~IPP,~R)  .48  .73  .3504  .87 

P(f|T,~ECM,-'IFP,~R  .19  .27  .0513  .13  . 

This  means  that  when  you  don't  get  a report  at  all,  you  are 
more  sure  than  you  were  before  that  it  is  an  enemy.  This  is 
because  the  likelihood  of  getting  no  report  is  48%  when  it's 
an  enemy,  and  only  19%  when  it's  a friend.  Why  is  this? 

Commander;  Well,  if  it's  an  enemy,  the  observers  on 
the  ship  aren't  vary  familiar  with  what  it  is,  so  they  would 
probably  wait  longer  to  be  sure  of  identifying  it.  On  the 
ot.ier  hand,  if  it's  one  of  our  planes,  they've  seen  so  many 
that  they  are  much  more  likely  to  identify  it  right  away. 

Consultant;  That  makes  sense.  Now,  as  a result  of  all 
these  information  inputs,  the  probability  of  the  aircraft 
being  an  enemy  has  risen  from  the  original  15%  to  87%; 
therefore,  you  should  shoot. 

As  you  probably  realized,  we  have  picked  out  only  one 
possible  result  for  each  information  input  for  this  analysis, 
while  in  reality,  there  are  two  possibilities  for  tip-off, 
ECM,  and  IFF,  and  three  for  the  visual  report  input.  To  do 
a complete  analysis  of  all  combinations  of  possibilities  and 
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al]  orders  of  occurrence  of  the  inputs,  computing  the  pos- 
terior odds  each  time  using  Bayes'  Theorem,  would  be  extremely 
time  consximlng.  Fortunately,  some  manipulation  of  Bayes' 
Theorem  and  the  use  of  a "log  odds"  chart  (see  Appendix  A) 
simplifies  the  problem.  Recall  that  Bayes*  Theorem  states 
that  the  posterior  odds  (the  revised  odds  In  light  of  new 
evidence)  equal  the  product  of  the  likelihood  ratio  for  that 
evidence  and  the  prior  odds  (the  odds  prior  to  receiving  the 
new  evidence) . This  form  can  lead  to  scale  problems  if  we 
attempt  to  display  the  impact  of  new  data  on  the  prior  odds  in 
a linear  (uniform)  graphical  form.  If  the  prior  odds  equal  1, 
and  the  likelihood  ratio  is  2:1,  then  the  posterior  odds  equal 
2;  but  with  prior  odds  of  100  (i.e.,  100:1)  and  the  same 
likelihood  ratio,  the  posterior  odds  equal  200.  On  an  odds 
scale,  the  apparent  degree  of  impact  of  the  datum  depends  on 
the  prior  odds. 

If  we  write  Bayes*  Theorem  in  logarithmic  form,  the  log 
of  the  posterior  odds  equals  the  log  of  the  likelihood  ratio 
plus  the  log  of  the  prior  odds: 

p(Ht1d)  p(d1h.)  P(Ht) 

log  « log  + log  =— 

P(H2|D)  P(d1H2)  P(H2)  . 

For  this  form  of  Bayes*  Theorem,  the  impact  of  a datum  is  to 
add  on  a fixed  amount  to  the  prior  odds,  whatever  tlie  value  of 
the  prior  odds.  Plotted  on  a log-odds  chart,  the  log  of  the 
likelihood  ratio  produces  the  same  increment  of  change  inde- 
pendent of  the  value  of  the  prior  odds.  As  long  as  the 
likelihood  ratio  for  each  datum  remains  the  same,  the  data  can 
be  considered  in  any  order  without  affecting  the  final  result. 

Let's  plot  the  results  of  our  analysis  on  a log-odds 
chart  (Figure  6-15)  along  with  some  information  we  have  from 
Figure  6-12.  That  showed  us  that  a P(e)  of  0.37  was  the 
crossover  point  between  the  SHOOT  and  NO  SHOOT  decisions. 

Conveniently,  a log-odds  chart  shows  probability,  in  this 
case,  P(e),  on  the  left  aide  and  the  corresponding  odds, 
P(e):P(f),  on  the  right  side.  We  start  with  our  prior  proba- 
bility of  0.15  (Point  A)  that  the  aircraft  is  an  enemy,  and 
plot  the  effect  of  each  input  in  serial  fashion  as  shown  by 
the  solid  line.  With  the  tip-off,  the  probability  of  enemy 
rises  to  41%  (Point  B) . The  lack  of  an  ECM  contact  doesn't 
change  the  probability  (Point  C)  since  the  odds  are  1:1  (log- 
odds  s 0) . No  IFF  return  increases  the  probability  to  73% 
(Point  D) , and  no  visual  report  raises  it  to  87%  (Point  E) . 

Suppose  we  had  not  had  the  tip-off  at  all.  Recall  that 
the  likelihood  of  a tip-off  if  the  aircraft  is  an  enemy  is 
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Figure  6- IB 


0.2;  that  leaves  0.8  for  the  likelihood  of  no  tip-off  given 
enemy ^ Also,  you  assessed  the  likelihood  of  a tip-off  given 
friend  as  .05,  so  the  likelihood  of  no  tip-off  if  the  air- 
craft is  a friend  must  be  0.95.  Thus,  when  no  tip-off  is 
received,  the  appropriate  likelihood  ratio  is  .80/. 95  = 

.84,  or  ill. 2.  Starting  with  our  prior  probability  of  0.15, 
or  odds  of  1:5.7,  no  tip-off  changes  the  prior  odds  to 
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These  odds  give  a posterior  probability  of  0,13.  So,  re- 
ceiving no  tip-off  reduces  the  probability  of  enemy  from  0.15 
to  0.13. 

Commander:  This  is  an  excellent  way  to  display  the 
analysis  because  it  shows  at  a glance  the  impact  of  the 
various  inputs.  It's  apparent  that  once  the  tip-off  occurred, 
the  odds  immediately  favor  the  decision  to  shoot,  while 
without  the  tip-off,  1 have  to  wait  for  the  IFF  input  before 
the  odds  favor  shooting.  In  fact,  if  I had  the  tip-off  that 
enemy  aircraft  were  in  the  air  prior  to  detecting  the  unknown 
aircraft  on  radar,  1 should  have  been  ready  to  shoot  at  once. 

Consultant:  Yes,  if  you  were  sure  !.\at  the  remaining 
information  inputs  would  remain  the  same.  We  didn't  analyze 
all  the  possibilities,  particularly  the  effect  of  a friendly 
IFF  return  or  visual  report  that  it  was  a friendly  aircraft. 
With  this  plot,  we  can  determine  those  impacts  quite  quickly. 

From  our  earlier  discussions,  the  probability  of  an  IFF 
return  if  it's  an  enemy  is  0.01  and  0.75  if  it's  a friend. 

This  is  a likelihood  ratio  of  1:75  against  it  being  an  enemy. 
Picking  the  log-odds  increment  off  our  scale,  we  can  plot  the 
results  as  shown  on  the  dotted  line  from  Point  C down  to  the 
bottom  of  the  chart  and  then  on  to  point  G in  Figure  6-15. 

Also,  the  probability  of  the  aircraft  being  an  enemy  if 
the  lookout  reports  it  as  friendly  is  0.02,  while  there  is  an 
80%  probability  it  is  friendly.  This  is  a likelihood  ratio  of 
1:40  against  it  being  an  enemy,  and  the  results  can  be  plotted 
as  the  dashed  line  from  Point  D to  Point  H,  on  Figure  6-15. 

In  both  cases,  the  impact  has  been  sufficient  to  change  your 
decision  from  SHOOT  to  NO  SHOOT. 

If  you  were  at  a different  location,  such  as  south  of 
Point  Bunker,  where  the  prior  probability  of  the  aircraft 
being  an  enemy  is  1%,  the  whole  analysis  would  simply  shift 
down  the  chart,  starting  at  P(e)  « 0.01,  or  1%.  This  is  true 
as  long  as  the  impact  of  the  information  inputs  does  not 
change  as  a result  of  the  different  location. 

Commander:  This  whole  session  has  been  very  enlight- 
ening, but  also  very  time  consuming.  In  the  case  of  USS 
HERRINGBONE,  there  wouldn't  have  been  enough  time  to  do  all 
this.  How  can  decision  analysis  help  in  a timely  manner  in  a 
tactical  situation? 

Consultant:  You're  right  in  what  you  say.  Of  course, 
the  purpose  of  this  particular  analysis  wasn't  to  help  you 
make  a decision  in  a real-time  situation,  but  was  rather  to 
analyze  a decision  which  had  already  been  made.  However, 
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these  tools  should  be  applicable  to  a real-time  situation  if 
you  can  identify  ahead  of  time  the  relevant  acts  and  events, 
then  develop  procedures  for  assigning  values  and  probabilities 
as  the  tactical  situation  changes.  What  you  really  need  is 
a computer  into  which  you  can  place  a prestructured  decision 
analysis  in  such  a way  that  the  expected  values  can  be 
computed  in  real  time  as  the  inputs  change.  It  is  fairly 
clear  that  the  values  will  change  very  slowly  with  time, 
while  the  probabilities  will  be  changing  quite  rapidly  in  a 
tactical  situation. 

Commander:  I have  a computer  available  in  my  Tactical 
Data  System,  which  is  used  to  display  the  tactical  lltuation. 

(At  this  point,  the  Task  Force  Commander  and  th«  Con- 
sultant engage  in  a discussion  of  how  the  Tactical  Data 
System  works  and  how  it  might  be  adapted  computing  and 
displaying  the  results  of  decision  analyait  in  a taotioal 
situation . ) 

Consultant:  Figure  6-16  is  a block  diagram  ahowing  hov 
the  Tactical  Data  System  could  be  used  for  dacia Ion -making 
in  a given  tactical  situation.  On  the  right  ara  all  tha 
inputs  that  are  either  already  available  to  you  or  that  you 
would  have  to  enter  into  the  computer  for  the  specific 
situation.  The  left-hand  inputs  are  the  dynamic  information 
from  the  sensor  systems  available  to  you  to  determine  the 
occurrence  or  non-occurrence  of  the  events  relevant  to  the 
decision.  These  inputs  would  cause  the  computer  to  update 
the  probability  associated  with  the  decision  in  accordance 
'|  with  your  prior  analysis. 

Values  refer  to  outcome  values  for  the  decision  diagram 
on  which  the  computer,  applying  branch  probabilities,  would 
operate  to  generate  expected  values  of  decisions  and  the 
crossover  point  at  which  your  decision  changes.  Your  Plans 
Department  would,  in  fact,  provide  the  changes  to  the  out- 
come values  you  have  previously  arrived  at  (Figure  6-8)  by 
responding  to  direction  from  higher  authority  or  from  your- 
self. For  example,  the  Seventh  Fleet  Commander,  under  whom 
you  operate  might  send  you  a message  stating,  "aircraft 
losses  must  be  reduced  by  20%. " Your  weighting  on  the 
equipment  dimension  would  change,  and  so  might  the  outcome 
values. 

The  readiness  inputs  on  the  right-hand  side  would 
permit  your  Weapons  Systems  Department  to  make  real-time 
changes  to  the  probability  of  kill.  Such  changes  could 
respond  to  either  materiel  or  personnel  values  as  they 
relate  to  readiness  and  could  change  from  watch  to  watch, 
for  instance.  The  last  input  on  the  right,  which  is  the 
priors,  woulc^  come  from  the  Intelligence  Department  and  is 
in  fact  the  i5%/85%  (enemy/ friendly)  assessment  that  you 
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started  off  with  as  your  prior  probabilities.  These  prior 
probabilities  are  developed  by  your  intelligence  department 
by  utilizing  the  all-source  data  base  and  applying  it  to 
the  herringbone's  actual  tactical  environment.  Obviously, 
these  inputs  on  the  right-hand  side  vary,  but  probably 
rather  infrequently. 

# 

Looking  at  the  inputs  on  the  left-hand  side  of  Figure 
6-16,  we  see  all  the  real-time  sensor  information  that  will 
vary  quite  dynamically  with  the  passage  of  time. 

Using  the  HERRINGBONE  analysis  as  an  example,  the 
detection  of  an  aircraft  by  radar  could  generate  a display 
like  that  shown  in  Figure  6-17.  The  target  is  given  an 
identification  number.  Bogie  26,  which  is  shown  on  the 
situation  plot  next  to  the  targef's  present  location.  The 
time,  06,  is  shown  on  the  outer  ring.  The  target  range  to 
closest  point  of  approach  is  100  nautical  miles,  and  the 
time  at  closes!  point  of  approach  is  16.  The  decision 
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window  shows  the  earliest  time,  11,  and  the  latest  time,  21, 
that  your  weapons  system  can  be  used  to  engage  the  target, 
if  the  decision  is  to  shoot.  These  window  times  will  vary 
with  maneuvering  but  are  dynamically  generated  by  the  Tactical 
Data  System.  At  time  06,  there  is  no  information  available 
from  the  sensors  about  Bogie  26  except  his  present  position, 
heading,  and  speed,  from  which  the  computer  has  generated 
the  plot  information.  The  probability,  based  on  prior 
intelligence,  that  it  is  an  enemy  has  not  changed  f::om  the 
prior  estimate  already  stored  in  the  computer.  Thus,  the 
Task  Force  Commander's  display  says  the  prior  odds  are  equal 
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to  the  posterior  odds,  or  a 15%  probability  of  an  enemy. 
Note  also  that  the  precomputed  probabilities  for  going  to 
general  quarters  and  for  freeing  weapons  are  displayed  for 
the  convenience  of  the  Commander  and  in  this  particular 
situation  are  5%  and  28%,  respectively. 

At  time  10,  you  receive  a tip-off  that  there  is  enemy 
aircraft  activity  in  the  area.  The  display  (Figure  6-18) 
indicates  that  a tip-off  has  been  received,  and  that  the 
posterior  probability  of  Bogie  26  being  an  enemy  after 
accounting  for  the  tip-off  information  has  increased  to 
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If  you  are  uncertain  about  why  such  a large  change  in 
probability  occurred,  the  computer  could  be  programmed  to 
display  the  likelihood  used  in  the  computation.  If,  in  the 
present  situation,  you  feel  that  your  prior  estimate  of 
likelihood  was  too  high,  you  could  change  it,  enter  the  new 
ratio  into  the  computer,  and  it  would  compute  and  display 
the  revised  probability. 

commander:  Since  1 know  in  advance  what  my  operations 
and  operating  areas  are,  it  would  be  reasonable  to  perform 
an  analysis  like  this  for  each  one  and  store  it  in  the 
computer.  I would  have  to  review  each  one  periodically  to 
be  sure  that  my  outcome  values  haven ' t changed  or  that  my 
probabilities,  based  on  previous  use  of  the  analyses, 
shouldn't  be  adjusted.  But  that  is  why  you  showed  me  Figure 
6-16,  to  illustrate  how  these  values  and  probabilities  can 
be  dynamically  updated  by  the  experts  of  my  staff,  from  all- 
source  data,  and  from  the  sensors  organic  to  my  task  unit. 

Now  1 can  see  how  decision  analysis  can  be  tactically  useful. 

Value  of  an  Information  Source 


Consultant;  Now  that  we  have  assessed  the  impact  of 
the  actual  information  on  the  probability  of  friend  or 
enemy,  we  can  use  this  seume  technique  to  calculate  the 
potential  value  of  a particular  source  of  information.  For 
example,  suppose  that  you  had  the  option  of  buying  the  IFF 
information  in  the  present  situation.  How  much  should  you 
be  willing  to  pay  for  it?  Should  you  be  willing  to  buy  it 
at  all? 

To  calculate  the  value  of  the  information,  we  will  use 
the  expected  value  of  the  basic  decision  a?  our  reference. 
Any  information  we  get  should  make  the  situation  better  than 
this,  so  we  can  take  the  expected  value  of  the  decision  with 
the  information  and  subtract  the  expected  value  for  the 
decision  without  the  information,  and  this  difference  is  the 
expected  value  of  the  information.  Now,  since  you  get  the 
IFF  information  after  you  have  already  received  the  tip-off, 
and  failed  to  get  the  ECM  signals,  the  basic  decision  to 
consider  is  the  decision  you  would  make  at  that  point  with- 
out any  IFF.  Let's  take  the  basic  decision  diagram,  and  put 
in  the  probabilities.  What  are  the  probabilities? 

Commander:  Let's  see,  we  have  received  the  tip-off, 

and  no  ECM  report,  so  according  to  the  chart,  the  proba- 
bilities at  that  point  are  .59  for  friend  and  .41  for  enemy. 

Consultant;  O.K.,  if  we  put  those  probabilities  into 
the  diagram,  and  compute  expected  values,  we  can  find  out 
what  the  expected  value  for  the  decision  without  information 
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a out.  IFF  is  worth.  (The  consultant  drew  the  diagram,  put 
i.  the  probabilities,  and  calculated  expected  values;  see 
Figure  6-19.) 

Conunaxider:  This  says  that  the  expected  value  for 
shooting  is  -3.8,  and  for  not  shooting  is  -4.8. 

Consultant:  Which  one  would  you  choose? 


Coiranander:  Naturally,  I would  choose  to  shoot,  but  I 
might  not  want  to  do  that  every  time. 

Consultant;  Well,  when  would  you  prefer  not  to  shoot? 

Commander:  When  I was  pretty  sure  it  was  a friend. 

Consultant;  Of  course,  that's  right,  but  I don't  see 
how  you  can  know  it's  a friend  without  more  information. 
Whenever  you  find  yourself  in  the  state  of  uncertainty  shown 
by  the  probabilities  in  Figure  6-19,  you  should  shoot.  The 
point  about  receiving  information  is  that  it  allows  you  to 
separate  situations  to  some  extent  into  two  categories.  In 
one  category,  the  probability  that  it  is  an  enemy  is  high 
enough  that  the  expected  value  of  shooting  is  greater  than 
the  expected  value  of  not  shooting,  so  you  shoot.  In  the 
other  category,  the  probability  that  it  is  an  enemy  is  low 
enough  that  the  expected  value  of  not  shooting  is  greater 
than  the  expected  value  of  shooting,  so  you  don't  shoot.  If 
there  is  no  circumstance  in  which  information  could  change 
your  decision,  then  the  information  isn't  worth  anything,  at 
least  with  respect  to  the  decision.  Now,  we  are  going  to  do 
just  that  kind  of  analysis  for  one  kind  of  information,  in 
this  case,  IFF  information. 

Commander;  O.K.,  how  do  we  set  it  up? 

Consultant;  The  first  thing  to  realize  is  that  we  now 
have  two  decisions;  the  first  is  whether  to  buy  information 
or  not,  and  the  second  is  whether  to  shoot  at  the  airplane 
or  not.  I'll  draw  the  diagram  like  this  (Figure  6-20).  The 
three  act  forks  at  tho  right  are  really  condensed  versions 
of  the  basic  decision  diagram.  They  are  called  the  primary 
decision  because  they  are  what  determines  the  information 
decision.  The  one  at  the  bottom  is  the  one  we  just  calcu- 
lated, the  basic  decision  diagram,  assuming  we  have  received 
the  tip-off  and  no  ECM  report,  so  we  can  put  the  values  of 
-3.8  and  -4.8  on  the  outcomes,  and  put  -3.8  in  the  box  as 
the  expected  value  of  the  decision.  Now,  the  other  two 
decision  diagrams  are  just  the  same,  except  that  they  incor- 
porate the  new  information  from  IFF  and,  therefore,  have  new 
probabilities.  Since  we  have  already  considered  what  happens 
when  there  is  no  IFF  return  signal,  let's  take  the  middle 
decision  diagram  first.  Now  everything  is  the  same  up  to 
the  point  where  the  probabilities  of  friend  and  enemy 
occur.  What  are  the  probabilities? 

Commander;  According  to  our  chart  (Figure  6-15) , the 
probabilities  after  the  ship  failed  to  get  an  IFF  return 
signal  were  .27  for  friend  and  .73  for  enemy. 
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Figure  6-20 

Consultant;  Right.  So,  if  we  incorporate  those  proba- 
bilities Into  our  decision  digram  (Figure  '6-21) , we  get 
expected  values  of  -0.55  for  shooting  and  -8.5  for  not 
shooting.  Now  what  if  the  ship  had  received  an  IFF  return 
signal?  What  is  the  probability  of  a friend  in  this  case? 

commander;  Since  we  didn't  consider  this  before,  I 
assume  we  have  to  use  Bayes'  Theorem  again. 

Consultant:  That's  right.  What  are  the  priors? 


197 


© © o o o o 


KILL 


-10 


Figure  6-21 

DECISION  DIAGRAM 


Commander!  They  are  .85  for  friend  and  .15  for  enemy, 
right? 

Consultant:  No,  remember  that  when  I aay  priors,  I 
mean  the  probabilities  that  you  have  prior  to  the  current 
information.  This  means  that  they  are  probably  posterior  to 
some  other  information. 

Commander:  Since  we  have  already  received  a tip- 
off  and'  there  was  no  ECM  reception,  the  probabilities  at 
this  point  are  .59  for  friend  and  .41  for  enemy. 
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Conaultant;  That's  right.  Now,  what  are  the  likeli- 
hoods? What  is  the  probability  of  getting  an  IFF  return 
signal  if  the  airplane  is  a friend? 

Commander:  1 said  before  that  it  was  .75,  and  that's 
about  right. 

Consultant:  And  if  it's  an  enemy? 

Commander:  Then  it's  only  1%. 

Consultant:  O.K.,  then  applying  Bayes'  Theorem,  we 

have: 


P(E| 

[T, -ECM,IPF) 

.41 

.01 

.0041 

.01 

P(Fj 

|t, ~ecm,iff) 

.59 

^ " w 

.75 

.4425 

.99 

We  can  put  these  probabilities  into  the  basic  decision 
diagram  as  we  did  before  (Figure  6-22) , and  the  expected 
values  are  -7.9  for  shooting,  and  -0.12  for  not  shooting. 
This  makes  sense,  since  we  are  very  sure  that  it's  a friend 
if  we  get  an  IFF  return  signal. 

Notice  that  these  last  two  decision  diagrams  make  up 
the  two  categories,  the  SHOOT  and  NO  SHOOT  categories,  so  ve 
know  already  that  the  information  Is  worth  something.  The 
only  thing  left  is  to  determine  how  much  it  is  worth.  If  we 
put  these  last  expected  values  into  our  information  decision 
diagram,  it  looks  like  this  (Figure  6-23) . 

Commander:  It  looks  to  me  as  though  we  could  decide 
right  now  to  purchase  the  IFF  equipment.  We  don't  need  to 
assign  probabilities  to  the  IFF  and  NO  IFF  branches  because 
the  expected  values  for  those  branches,  -0.12  and  -0.55, 
are  each  smaller  than  the  expected  value  for  not  purchasing 
the  IFF,  -3.8.  No  matter  what  probabilities  we  assign,  the 
expected  value  of  purchasing  the  IFF  will  be  smaller  than 
-3.8. 


Consultant:  That's  true,  but  remember  that  we  haven't 
included  the  cost  of  the  IFF  in  our  analysis.  I doubt  that 
you  would  purchase  the  equipment  if  it  cost  610  million;  the 
information  wouldn't  be  worth  the  cost.  Your  observation  is 
correct  only  if  the  cost  of  the  IFF  is  less  than  the  expected 
value  of  the  information  it  provides.  We  have  to  complete 
the  analysis  so  we  can  determine  the  expected  value  of  the 
information.  What  do  we  need  to  know  in  order  to  finish  the 
diagram? 

Commander:  There  is  an  event  fork  here,  and  the  two 
branches  say  IFF  and  no  IFF. 


199 


FRIEND 


SHOOT 


NO  KILL 


DECISION 


FRIEND 


Figure  6-22 


Consultant:  Yes,  there  Is  some  probability  that  you 
will  get  an  IFF  return  signal.  As  you  said  earlier,  the 
probability  of  getting  an  IFF  return  is  high  from  a friend 
and  low  from  an  enemy.  But  we  need  to  find  the  probability 
of  getting  an  IFF  return  without  having  to  specify  whether 
the  airplane  is,  in  fact,  a friend  or  an  enemy.  To  do  this 
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SHOOT 


let's  take  a small  detour,  and  draw  a probability  diagram 
(Figure  6-24)  to  represent  the  way  you  think  about  these 
uncertainties . 

First,  the  airplane  is  either  a friend  or  an  enemy.  If 
it's  a friend,  then  there  is  either  a return  signal  or  there 
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Figure  6-24 


iB  not.  Similarly,  if  it  is  an  enemy,  there  is  either  a 
signal  or  not.  Now,  you  have  previously  assessed  the  proba- 
bilities in  this  diagram  relating  to  the  IFF  return,  and  we 
have  used  Bayes'  Theorem  to  determine  the  probabilities  on 
the  branches  of  the  left  fork  after  having  received  a tip- 
off  and  no  ECM,  right? 

Commander:  Yes,  the  probability  of  a friend  is  0.59, 
and  the  probability  of  enemy  is  0.41.  if  it's  a friend,  the 
probability  of  IFF  is  0,75,  and  the  probability  of  no  IFF  is 
0.25.  If  it's  an  enemy,  the  probability  of  IFF  is  .01,  and 
I the  probability  of  no  IFF  is  0.99. 

Consultant:  O.K. , the  probability  of  IFF  is  just  the 
probability  of  IFF  if  it's  a friend  times  the  probability 
that  it  is  a friend,  plus  the  probability  of  IFF  if  it's  an 
enemy  times  the  probability  that  it's  an  enemy. 

P(IFF)  - P(IFFIf)  P(P)  + P(IFFIE)  P(E) 

- (0.75) (0.59)  + (0.01)  (0.41) 

» 0.4425  + 0.0041 
» 0.45 

(Note  that  we  are  applying  both  the  multiplication  and 
addition  rules  for  probability  as  explained  in  Chapter  4 . ) 
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The  probability  of  no  IFF  is  (.25)  x (.59)  + (.99)  x (.41)  = 
.55.  Now  we  can  put  these  probabilities  into  our  infor- 
mation diagram  and  calculate  the  expected  value  for  buying 
IFF  information  (Figure  6-25).  It  comes  out  tf'  be  -0.4.  So 
should  you  buy  IFF  information  or  not? 
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Conunander:  As  you  said  before , that  depends  on  how 
much  the  IFF  costs. 

Consultant:  What  if  I offered  to  sell  you  the  Infor- 
mation at  a cost  of  3.3  million  dollars.  Should  you  buy  it? 

Commander:  Let  me  think.  If  the  information  cost  me 
3.3  million,  and  the  expected  value  of  the  decision  with  the 
information  is  worth  -0.4  million,  then  the  total  expected 
cost  of  making  the  decision  with  the  information  is  0.4  + 

3.3  “ 3.7  million,  right? 

Consultant:  That's  right. 

Commander:  Then  I would  buy  the  information,  because 
it  still  has  a lower  cost  than  the  3.8  million  expected  cost 
for  deciding  without  the  information. 

Consultant:  You  have  the  whole  concept  of  the  value  of 
information  in  a nutshell.  What  is  the  value  of  the  infor- 
mation? 

Commander:  It  must  be  3.4  million  dollars,  because  I 
would  buy  the  information  if  it  cost  less  than  the  amount, 
and  I wouldn't  buy  it  if  it  cost  more. 

Consultant:  That's  exactly  right. 

Commander:  Does  this  mean  that  I should  be  willing  to 
pay  3.4  million  dollars  to  have  IFF  equipment  installed  on 
my  ship? 

Consultant:  Not  quite.  This  is  only  the  value  of  IFF 
in  this  situation.  In  order  to  evaluate  IFF  in  general,  you 
would  have  to  consider  other  scenarios,  especially  ones 
which  are  more  likely  to  occur  than  this  one.  In  addition, 
you  would  have  to  consider  other  uses  of  IFF,  such  as  con- 
trolling traffic  among  your  own  aircraft,  identification, 
and  so  forth. 

Value  of  Information  vs.  Value  of  Weapons  Systems 


Consultant:  Thus  far,  we  have  talked  about  the  impact 
of  information  on  the  primary  decision  to  shoot  or  not  to 
shoot.  We  have  demonstrated  that  information  changes  the 
expected  value  of  the  decision  by  changing  the  probability 
that  the  incoming  unidentified  aircraft  is  a friend  or  an 
enemy.  Notice,  however,  that  the  probability  that  the 
incoming  aircraft  is  a friend  or  an  enemy  is  not  the  only 
probability  which  can  affect  the  expected  value  of  the 
primary  decision.  The  likelihood  that  you  will  hit  the 
incoming  aircraft,  given  that  you  decide  to  shoot  at  it, 
will  also  cause  the  expected  value  to  vary.  To  show  this. 
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let's  fix  the  probability  of  enemy  at  87%  and  the  probability 
of  a friend  at  13%  and  vary  the  kill  probability  of  the 
weapon  system.  In  Figure  6-26,  the  kill  probability  is  set 
rt  zero,  the  probability  of  not  killing  is  set  at  100%. 

Notice  here  that  the  expected  value  of  shooting  is  a -5.9, 
the  expected  value  of  not  shooting  is  a minus  10.2.  Those 
choices  are  pretty  unappealing,  but  the  preferred  decision 
is  to  shoot. 


On  the  other  hand,  if  the  kill  probability  were  moved 
to  the  other  extreme,  so  that  you  could  be  100%  certain  of 
hitting  an  airplane  you  shoot  at,  then,  as  Figure  6-27 
shows,  the  expected  value  of  shdoting  goes  to  +3.7,  the 
expected  value  for  not  shooting  remaining  at  -10.2.  Thus, 
the  value  of  increasing  the  kill  probability  from  zero  to 
100%  is  the  difference  between  the  expected  values  of  the 
decisions  to  shoot,  or  3.7  -(-5.9)  = 9.6  million  dollars. 


Commander:  As  I understand  what  you've  just  done, 
you've  said  that  the  value  of  the  weapon  system  is  equal  to 
the  difference  in  expected  values  of  the  decisions  which  would 
be  made,  given  that  the  kill  probability  characterizing  that 
weapon  system  has  been  changed.  This  seems  to  be  the  same 
procedure  we  used  to  calculate  the  value  of  Information. 

Consultant:  That's  right.  Just  as  the  value  of  infor- 
mation depended  upon  the  assigned  kill  probability,  so  the 
value  of  changing  the  kill  probability  (the  value  of  the 
weapon  system)  depends  upon  the  probabilities  assigned  to 
friend  and  enemy.  Obviously,  if  you  were  completely  certain 
that  the  airplane  was  friendly,  then  you  wouldn't  pay  anything 
to  increase  your  kill  probability  since  you  wouldn't  be 
shooting . 

Commander:  I get  the  feeling  that  there  must  be  a 
trade-off  between  purchasing  information  and  improving  the 
weapon  system.  It  looks  to  me  like  different  combinations  of 
probability  of  friend  or  enemy  and  probability  of  kill  or  no 
kill,  would  result  in  the  came  expected  value.  It  would  seem 
to  me  that  one  way  to  display  this  would  be  by  plotting  those 
combinations  of  P(e)  and  P(k}  which  gave  the  same  expected 
value.  I can  imagine  doing  this  for  different  expected  values 
so  I'd  end  up  with  something  like  a series  of  contour  lines. 
Then  if  someone  offered  me  the  opportunity  to  purchase  infor- 
mation which  would  change  my  prior  probability  a certain 
amount  or  to  improve  my  kill  probability,  I could  determine 
exactly  what  the  trade-off  was. 

Consultant:  That's  almost  right.  I think  from  the  way 
you  described  your  approach,  however,  you've  overlooked  a 
subtle  point.  That  is,  you  won't  know  in  a given  situation 
what  kind  of  information  you're  going  bo  get.  As  I understand 
what  you  were  saying,  you  envisage  making  a trade-off  between, 
say,  receiving  a positive  IFF  report,  and  increasing  P(k)  from 
70  to  75%.  The  problem  with  this  is  that  you  won't  know  in 
advance  exactly  what  infonnation  you're  going  to  receive.  You 
might  get  a positive  IFF  report  which  would  decrease  the 
probability  of  enemy,  but  on  the  other  hand,  you  might  get  a 
negative  IFF  report  which  would  increase  that  probability. 

Commander:  You're  right.  I'd  forgotten  about  that.  I 
remember  now  that  when  we  were  doing  the  value  of  information 
analysis,  we  had  to  consider  the  possibility  of  getting  both 
kinds  of  reports  and  then  weight  the  impact  of  each  report  by 
the  probability  that  we'd  get  that  report. 

Consultant:  That's  right  and  that's  almost  exactly  the 
procedure  we  could  use  here.  The  calculations  involve  no 
new  concepts,  and  since  they  are  a little  involved,  we  won't 
do  them  just  now.  However,  eventually  we  could  draw  curves 


that  would  enable  you  to  compare  the  relative  values  of 
better  information  and  increased  weapons  capability. 

Commander;  That's  really  a very  clever  trade-off.  It 
seems  to  me  that  one  of  the  biggest  problems  we  have  had  in 
the  past  is,  in  fact,  trying  to  show  the  value  of  information 
versus  the  value  of  weapons  systems,  and  this  looks  like  a 
very  good  way  to  do  it. 

Consultant:  Now  we've  completed  the  analysif  of  the 
decision  problem  faced  by  the  commander  of  USS  HERRINGBONE.  I 
think  the  major  benefit  of  any  decision  analysis  comes  through 
the  increased  understanding  of  the  problem  afforded  by  the 
analysis.  When  this  occurs,  the  nunibers  that  went  into  the 
analysis  become  less  important.  You  might  say  that  we've 
taken  a quantitative  approach  in  order  to  get  a better  quali- 
tative feel  for  the  problem. 

Commander;  The  sensitivity  analyses  were  particularly 
informative.  You  quickly  get  a sense  of  what  is  important  and 
what  isn't.  I only  wish  we  could  have  done  more  of  them. 

Consultant:  We  could  have  done  a great  many  more,  very 
quickly,  If  we  had  used  a computer.  That's  a major  role  that 
is  emerging  for  the  new  generation  of  desk-top  computers. 

Using  an  interactive  program  for  computer-aided  decision 
analysis,  you  could  do  on  the  computer  everything  we  did  here, 
and  more.  You  use  the  computer  as  an  extension  of  your 
intellect;  it  does  the  messy,  complex,  and  tedious  chores 
while  you  supply  the  judgments.  In  addition,  it  can  check  on 
the  consistency  of  your  judgments,  inform  you  of  inconsistencies, 
and  give  you  the  opportunity  to  correct  errors.  Because 
judgment  is  an  important  element  in  most  important  or  difficult 
decisions,  it  is  neither  possible  nor  desirable  to  hand  over 
the  decision-making  process  to  the  computer.  But  the  computer 
can  be  a useful  tool  for  the  decision  maker;  that's  why  we 
talk  about  computer-aided  decision-making. 

Commander;  We've  covered  a lot  of  ground,  and  I'm  not 
sure  I'll  remember  all  this. 

consultant:  Because  decision  analysis  is  as  much  an 
approach  as  it  is  a set  of  techniques,  it  usually  takes 
further  work  and  attempts  to  apply  the  approach  before  the 
ideas  settle  comfortably.  Eventually,  however,  you  will  find 
that  decision  analysis  provides  a useful  and  natural  way  of 
thinking  about  problems  involving  risk  and  uncertainty. 
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APPENDIX  A 


INFERENCE  FROM  EVIDENCE;  BAYES'  THEOREM 


An  analyst,  especially  one  in  the  field  of  intelligence, 
is  often  required  to  work  with  data  that  are  incomplete  or 
possibly  unreliable.  Since  he  must  use  his  judgment  to 
evaluate  the  worth  of  that  material,  he  needs  to  develop  a 
technique  that  will  help  him  to  give  his  best  judgment.  This 
appendix  describes  such  a method;  it  is  based  upon  Bayes' 
Theorem  and  allows  the  analyst  to  combine  new  information  with 
his  prior  assessment  of  the  probability  of  an  occurrence. 

Consider  the  following  problems.  In  each  there  is  a need 
to  revise  the  judgments  of  the  probabilities  of  events: 

1.  A Russian  submarine  has  been  sighted  leaving  the 
Mediterranean  Sea,  and  the  analyst  needs  to  infer 
whether  or  not  it  was  a uclear  submarine.  Pre- 
viously, the  analyst  felt  there  war.  a 60%  proba- 
bility it  was  nuclear.  How  can  he  revise  this 
probability  in  light  of  the  fact  that  the  submarine 
seems  to  be  following  a straighb  track  rather  than 
taking  evasive  action? 

2.  The  flag  officer  of  an  aircraft  carrier  is  concerned 
with  an  unidentified  plane  on  a straight  course  to 
his  ship.  Before  attempted  communication,  there  is 
a 50:50  chance  it  is  friendly.  If  the  plane  refuses 
radio  acknowledgment,  is  the  probability  likelihood 
of  its  being  an  enemy  airplane  now  high  enough  to 
warrant  attempting  to  shoot  it  down? 

3.  During  the  eight  months  leading  up  to  the  Soviet 
Union's  invasion  of  Czechoslovakia,  how  did  the 
occurrence  of  new  intelligence  data  affect  the 
probability  of  an  invasion?  (This  problem  will  be 
examined  in  depth  later  in  the  appendix.) 

An  analyst  can  of  course  make  direct  assessments  of  any 
of  the  probabilities  called  for  in  the  above  examples.  It 
will  often  happen,  however,  that  he  feels  more  comfortable  or 
more  confident  with  a somewhat  different  ^ype  of  assessment. 
For  example,  it  may  be  more  natural  for  him  to  address  the 
question  "What  is  the  likelihood  that  a nuclear  submarine 
would  travel  in  a straight  track  rather  than  take  evasive 
action?"  inste.-»d  of  the  required  probability  which,  in  a 
sen^e,  is  the  other  way  around:  "What  is  the  probability  it 
is  nuclear  if  we  know  it  is  following  a straight  track?" 
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One  o±  the  very  important  roles  of  probability  theory 
is  that  it  often  allows  the  analyst  to  deduce  in  a logical 
fashion  the  probabilities  he  wants  from  probabilities  and 
likelihoods  already  at  hand.  For  example,  probabilities 
"posterior'*  to  a new  piece  of  evidence  can  be  deduced  from 
"prior"  probabilities  and  from  the  likelihood  of  observing 
the  evidence  given  the  hypothesis  is  or  is  not  true. 

We  ask  the  reader  to  bear  with  us  for  a few  paragraphs 
while  we  introduce  some  basic  concepts  and  notations  which 
underlie  a particularly  useful  procedure  for  updating  uncer- 
tainty as  new  information  is  received. 

Bayes ' Theorem 

All  of  the  problems  cited  above  can  be  solved  through 
the  use  of  Bayes*  Theorem,  a formal,  optimal  rule  for  the 
revision  of  probabilities  in  the  light  of  new  evidence.  The 
actual  formula  is  fairly  simple  and  can  be  derived  from  the 
rules  of  probability  discussed  in  Chapter  3. 

Firs'^,  it  is  necessary  to  define  three  inputs  to  Bayes* 
Theorem.  (1)  A prior  (or  unconditional)  probability  of  a 
hypothesis,  H,  is  the  probability  an  assessor  assigns  to  it 
prior  to  receiving  any  information  and  is  represented  as 
P(H).  (2)  A likelihood  of  a datum  is  assigned  given  know- 

ledge of  some  hypothesis , H,  and  is  represented  as  P(D|H). 

(3)  A joint  probability  is  the  probability  of  two  events 
both  occurring,  that  is,  the  probability  that  both  the 
hypothesis  is  true  and  that  a datum  is  observed.  This 
probability  is  represented  as  P(H&D). 

These  three  terms  can  be  represented  conveniently  in  a 
probability  diagram  as  shown  in  Figure  A-1:  P(H)  ■ .2 
represents  an  assigned  probability  of  . 2 that  the  hypothesis 
is  true,  unconditional  on  the  data;  P(d{h)  >=  .6  represents  an 
assigned  likelihood  of  .6  that  a particular  datum  would  be 
observed  if  the  hypothesis  were  true;  and  P (D&H)  = .12 
represents  a probability  of  .12  that  both  the  hypothesis  is 
true  and  the  datum  is  observed.  [Note:  P(~D)  is  the  proba- 
bility of  the  datum,  D,  not  occurring.] 

When  these  three  quantities  are  defined  in  general 
terms,  they  can  be  used  to  construct  Bayes*  Theorem.  First, 
the  definition  of  the  likelihood  of  an  event  D,  or  datum, 
occurring,  given  hypothesis  H is, 

P(d1h)  = -p-jjjy---  . (1) 

Equation  (1)  also  holds  for  the  reverse  case  of  the  proba- 
bility of  the  hypothesis  given  the  datum, 

P(H|D)  = . (2) 
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PIDftH)  » .12 


By  combining  equations  (1)  and  (2) i we  derive  the  basic  form 
of  Bayes'  Theorem, 


P(h1d)  - . (3) 

In  review, 

P(0)  - Probability  of  the  datum  occurring. 

P (D I H)  > Likelihood  of  the  datum,  given  the  hypo- 

thesis Is  true. 


P{H) 


Prior  probability  of  hypothesis  H,  based 
on  all  information  about  H which  Is 
available  before  the  new  datum  D is 
learned. 

Posterior,  or  revised,  probability  of  the 
hypothesis  occurring,  given  that  the 
datum  has  been  observed. 


For  the  case  of  two  hypotheses.  Hi  and  H2  (i.e..  Hi  ■ the 
Russian  submarine  is  nuclear,  and  H2  *■  the  Russian  submarine 
is  non-nuclear),  Bayes'  Theorem  can  be  written  twice: 


P(H^lD) 


P(dIh^)  P(H^) 


(4) 


P(D) 
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By  dividing  one  equation  by  the  other,  the  P(D)'b  in  the 
denominators  cancel  out  and  leave  the  following: 

P(HilD)  P{DlHi)  P(Hi)  ^ (gj 

P(H2lD)  P{D|H2)  P(H2) 

The  ratio  far  to  the  right  of  the  equal  sign  in  equation  (6) 
is  the 

P(Hi)  (7) 

Prior  Odds  - ^ ^ ' 

P(H2) 

It  is  equal  to  the  probability  that  H][  is  true  divided  by 
the  probability  that.  H2  is  true  prior  to  observing  the 
evidence.  The  ratio  immediately  to  tKe  right  of  the  equal 
sign  in  equation  (6)  is  called  the  likelihood  ratio  (LR) , or 

• «» 

P(D|H2) 

It  is  equal  to  the  probability  that  the  datum  would  have 
occurred  if  is  true  divided  by  the  probability  that  it 
would  have  occurred  if  H2  is  true.  The  ratio  to  the  left  of 
the  equation  is  the  resulting  posterior  odds,  the  odds  in 
favor  of  H],  over  H2  after  the  datum  has  been  observed: 

P(HilD) 

Posterior  Odds  - , or  (9) 

P(H2|D) 

Posterior  odds  «•  LR  x Prior  Odds.  (10) 

Equation  (10)  can  now  be  used  to  revise  the  odds  from 
prior  to  posterior  as  a function  of  the  information  in  the 
observed  datum.  Because  of  the  kind  of  thinking  required 
to  use  this  method,  Bayes'  Theorem  is  often  called  the 
theorem  of  inverse  probabilities.  To  go  from  prior  odds  to 
the  posterior  odds,  it  is  necessary  to  assume  first  that  Hi 
is  true  and  then  assume  that  H2  is  true,  and  to  ask  how 
likely  it  is  that  the  datum  would  be  observed  under  each 
assumption.  The  answers  to  that  question  are  specified  in 
the  likelihood  ratio.  To  the  extent  that  the  datum  is  more 
likely  if  Hi  is  true,  the  posterior  odds  are  greater  than 
the  prior  odds.  To  the  extent  that  the  datum  is  more  likely 
under  H2,  the  reverse  is  true. 
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Czechoslovakia  Case 


Since  Bayes*  Theorem  can  be  used  to  update  existing 
informationf  consider  the  following  problem  in  which  new  data 
are  periodically  received:  Assume  that  today  is  January  6/ 
1968,  and  the  following  article  appeared  in  yesterday's  New 
York  Times  newspaper: 

Vienna,  Jan.  5,  1968.  Antonin  Novotny  has  been  ousted  as 
the  Czechoslovak  Communist  Party  chief,  the  Czecn  press 
agency  CTK  confirmed  tonight.  However,  he  will  remain  as 
President  of  the  country.  The  Yugoslav  press  agency 
Tanyug  first  reported  the  ouster  of  Mr.  Novotny  as  First 
Secretary  of  the  Party  and  the  Czechoslovak  government 
confirmed  it  by  distributing  photographs  of  Alexander 
Dubcek  and  identifying  him  as  the  new  Party  leader. 

Now  that  the  Soviet  favorite,  Antonin  Novotny,  has  been 
replaced  by  a liberal,  Alexander  Dubcek,  the  chance  for  a 
Soviet  invasion  has  become  high  enough  that  the  situation  is 
worth  considering.  (Mote:  Although  the  invasion  of  August 
20,  1968,  is  now  history,  assume  this  is  a future  event  that 
may  or  may  not  occur.)  The  job  of  the  U.S.  analyst  is  to 
follow  the  situation  for  the  next  year  and  to  update  peri- 
odically the  likelihood  of  an  invasion  as  new  intelligence 
data  are  received.  For  simplification,  the  only  source  of 
information  will  be  actual  newspaper  articles  from  the  New 
York  Times,  i.e.,  ptiblic  information. 

Consider  the  two  hypotheses:  The  first  hypothesis,  Hi, 
is  that  the  Soviet  Union  will  invade  Czechoslovakia  within  one 
year;  the  second  hypothesis,  H2,  is  that  it  will  not  invade 
Czechoslovakia  within  one  year.  The  analyst's  first  step  is 
to  estimate  today,  January  6,  1968,  the  likelihood  of  an 
invasion  occurring  within  one  year.  Although  a new  Communist 
Party  chief  is  now  in  power  in  Czechoslovakia,  it  is  unlikely 
that  the  USSR  will  consider  an  invasion  unless  Dubcek  starts 
to  make  radical  changes.  In  this  case,  the  analyst  may  deter- 
mine that  there  is  a one  to  20  chance  of  Hy  being  true,  that 
is,  the  prior  odds  are  given  by 

P(H^)  1 

P(H2)  20 

The  easiest  way  of  displaying  the  impact  of  new  data 
on  the  prior  odds  is  through  the  use  of  a "log  odds"  chart. 
Consider  once  again  Bayes'  Theorem  as  written  in  equation 
(6) . The  posterior  odds  equal  the  product  of  the  likelihood 
ratio  and  the  prior  odds.  If  the  prior  odds  equal  1 and  the 
likelihood  ratio  is  2:1,  the  posterior  odds  equal  2.  But  if 
the  prior  odds  equal  100  and  the  likelihood  ratio  is  2:1, 
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the  posterior  odds  equal  200.  Thus,  on  an  odds  scale,  the 
apparent  degree  of  impact  of  the  datum  depends  upon  the  value 
of  the  prior  odds.  Any  given  likelihood  ratio  will  result  in 
greater  movement  of  the  odds  when  the  prior  odds  are  already 
high  than  when  the  prior  odds  are  low. 

This  problem  does  not  occur  when  Bayes*  Theorem  is 
written  in  logarithmic  form.  When  both  sides  of  equation  (6) 
are  transformed  into  logarithms,  the  log  of  the  posterior  odds 
equals  the  log  of  the  prior  odds  plus  the  log  of  the  likeli- 
hood ratio: 

P(H,  |D)  P(DiH,  ) P(H,) 

log  ± - log  =—  + log =—  . (11) 

P(H2|D)  PCDiHj)  PCHj) 

Whatever  the  prior  odds,  the  impact  of  a datum  is  to  add  on  a 
fixed  amount  to  the  prior  odds.  For  example,  assume  that  the 
log  of  the  likelihood  ratio  is  0.5.  If  the  prior  log  odds 
equal  0.0,  then  the  posterior  log  odds  equal  4-0.5.  This 
additive  version  of  Bayes*  Theorem  yields  a simple  graphical 
display  of  posterior  odds. 

Consider  the  chart  in  Figure  A-2.  The  Y-axis  is  scaled 
In  log  odds,  but  the  right  vertical  axis  Is  marked  in  odds, 
the  left  vertical  in  probabilities.  The  horizontal  axis  is 
scaled  in  time.  Both  hypotheses  are  equally  likely  in  the 
middle  of  the  Y-scale  where  the  odds  are  1:1.  becomes 
increasingly  likely  as  the  scale  goes  up,  and  H2  becomes 
Increasingly  likely  as  the  scale  moves  down.  According  to  the 
previous  estimate,  the  prior  odds  are  1:20  and  are  represented 
on  the  following  log  odds  chart, 1 Figure  A-2. 

Datum  #1:  The  first  datum  concerning  the  hypothesis  occurs  on 
February  22: 

Prague,  Feb.  22.  The  Czechoslovak  Communist  Party  began 
celebrating  today  its  20th  Anniversary  in  power  amid 
indications  that  some  members  were  uneasy  about  the 
presence  of  their  most  prestigious  foreign  guest, 

Leonid  I.  Brezhnev,  the  Soviet  Party  Leader  . . . Some 
Czechoslovaks  are  concerned  that  Mr.  Brezhnev  may  be  here 
to  put  pressure  on  Alexander  Dubcek,  the  47-year-old 
Slovak  v^’iO  replaced  the  Czech  party  leader,  Antonin 
Novotny,  less  than  two  months  ago. 




The  log  odds  charts  used  in  this  Appendix  were  drawn  up  tc 
facilitate  the  use  of  the  odds  scale  on  the  right-hand  side 
of  the  chart.  The  probabilities  on  the  left-hand  scale 
have  been  approximated  to  the  nearest  integral  1%,  and  were 
included  only  for  the  reader *b  reference. 
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Figure  A-2 
PRIOR  ODDS 

Now  that  a new  piece  of  datum  is  available,  the  analyst 
needs  to  update  hla  earlier  prediction  from  the  prior  odds 
of  1:20.  He  must  decide  If  this  datum  has  any  bearing  on 
Hi,  and  If  It  does,  whether  it  favors  Hj  or  H2,  and  to 
what  extent.  Since  the  new  posterior  odds  are  equal 
to  the  LR  times  the  prior  odds,  we  need  only  to  deter- 
mine the  LR  to  calculate  the  revised  odds.  Because 
Brezhnev  may  be  In  Czechoslovakia  to  put  pressure  on 
Alexander  Dubcek,  this  datum  slightly  favors  the  hypothesis 
of  an  Invasion.  In  this  case,  the  analyst  assesses  a proba- 
bility of  .80  that  Brezhnev  would  be  there  If  an  Invasion 
were  planned,  and  a .50  probability  that  he  would  be  there 
if  there  were  to  be  no  Invasion,  therefore  the  likelihood 
ratio  of  the  first  datum  Is  .80/. 50  ■ 1.6:1.  With  this  value 
and  the  prior  odds,  we  can  compute  the  posterior  odds: 


Posterior  = LR  x Prior 

- 1.6  1 


“ rh  • 

The  updated  posterior  odds,  1:12.5,  are  shown  on  the  chart 
in  Figure  A- 3,  the  dashed  line  from  the  prior  odds  to  the 
posterior  odds  indicating  that  the  odds  have  been  revised. 


1 


\ 


I 
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Note  that  it  is  only  the  likelihood  ratio  which  affects 
the  posterior  odds.  It  does  not  matter  what  value  the  like- 
lihoods have  as  long  as  their  ratio  is  unaffected. 

This  has  an  important  practical  implication  for  work, 
especially  in  the  field  of  intelligence,  because  it  permits 
the  analyst  to  avoid  awkwardness  in  specifying  exactly  what 
the  datum  is  and,  therefore,  its  precise  probability  condi- 
tional on  the  alternative  hypotheses.  Data  can  be  defined 
loosely  or  precisely.  Datum  #1,  for  example,  could  be  inter- 
preted as  meaning  the  exact  wording  of  the  newspaper  report. 

In  that  case,  the  probabilities  appearing  in  the  likelihood 
ratio  would  be  infinitesimally  small,  but  it  might  still  be 
reasonable  to  assess  their  ratio  to  each  other,  i.e.,  the 
ratio  of  1.6  to  1.  On  the  other  hand,  the  datum  might  be  more 
loosely  defined  in  a way  that  simply  captured  the  flavor  of 
the  communique.  In  this  case  the  likelihoods  would  be  larger, 
but  in  all  probability,  the  ratio  would  still  be  very  close  to 
1.6  to  1.  In  circumstances  like  this — which  are  very  common— 
when  the  likelihood  ratio  is  not  very  sensitive  to  how  the 
datum  is  defined,  there  is  no  need  to  be  overly  precise.  This 
is  a marked  practical  convenience. 

Datum  #2} 

Berlin,  April  21.  Marshal  Ivan  I.  Vakubovsky  of  the 
Soviet  Union,  the  commander  of  the  Warsaw  Pact  Military 
Alliance,  arrived  in  East  Berlin  today  for  talks  with 
Walter  Ulbricht,  First  Secretary  of  the  Communist  Party, 
on  measures  to  reinforce  the  Alliance's  defense  system. 
The  marshall  came  from  Warsaw,  where  he  had  conferred 
with  Polish  leaders. 

In  this  case,  if  the  Soviet  Union  were  planning  an 
invasion,  there  is  an  assessed  95%  probability  that  Marshal 
Yakubovsky  would  meet  with  the  different  members  of  the 
Warsaw  Pact;  and  there  is  an  assessed  probability  of  40%  that 
he  might  be  meeting  with  them  on  other  routine  matters  if  no 
invasion  were  planned.  The  likelihood  ratio,  2.4:1,  for  the 
second  datum  also  favors  Hx#  giving  new  posterior  odds  that 
can  be  drawn  on  the  chart.  Figure  A- 4. 

Posterior  * LR  x Prior 
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Datum  #3; 

Prague,  May  3,  It  was  disclosed  tonight  that  Alexander 
Dubcek,  the  new  Communist  Party  chief,  had  flown  to 
Moscow  for  meetings  with  the  Soviet  Government. 

Datum  #3  only  informs  the  analyst  that  Dubcek  is  in 
Moscow  meeting  with  the  Soviet  Government.  It  is  up  to  the 
analyst  to  use  his  knowledge  of  the  situation  to  determine 
if  this  is  just  a routine  meeting  to  discuss  trade  or  economic 
matters,  or  if  it  is  an  indication  of  a serious  rift  con- 
cerning communist  ideology.  The  analyst  may  feel  there  is 
an  80%  probability  Datum  #3  would  occur  if  there  is  to  be  an 
invasion  and  a 40%  probability  Datum  #3  would  occur  without 
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an  invasion  being  planned.  The  effeot  of  the  new  likelihood 
ratio/  2.0:1,  is  displayed  in  Figure  A-5. 

•j 

Note  that  when  using  Bayes ' Theorem  and  log  odds  charts , j 

it  is  not  necessary  to  take  the  data  in  order.  The  present  f 

posterior  odds  of  1:2.6  would  have  been  determined  regard-  ^ 

less  of  the  order  in  which  Data  #1,  #2  and  #3  had  been  j 

considered  as  long  as  the  three  likelihood  ratios  could 
remain  the  same.  The  order  must  be  considered  only  if  the  i 

component  likelihood  ratios  change  for  different  orders.  ' j 


Datum  #4; 


Prague,  May  24.  The  armed  forces  of  the  Warsaw  Pact  will 
hold  exercises  on  Czechoslovak  and  Polish  territory  next 
month,  Czechoslovakia  announced  tonight.  The  statement 
distributed  by  the  Ceteka  press  agency,  said  that  the 
exercises  would  be  under  the  command  of  Marshal  Ivan  F. 
Yakubovsky  of  the  Soviet  Union,  Commander-ln-Chlef  of  the 
forces  of  the  Warsaw  Pact.  The  military  alliance  of  the 
Eastern  Bloc  consists  of  the  Soviet  Union,  Bulgaria, 
Czechoslovakia,  East  Germany,  Poland  and  Rumania. 

This  new  datum,  considered  after  the  occurrence  of  the 
other  previous  events,  is  a very  strong  one  and  now  clarifies 
Marshal  Yakubovsky 's  actions  in  East  Berlin  on  April  21 
(Datum  #2) . The  analyst  assessed  that  the  event  was  certain 
to  occur  if  an  invasion  were  imminent,  but  the  probability  was 
only  10%  if  no  invasion  were  planned.  The  resulting  LR  of 
10:1  is  now  added  to  the  log  odds  chart  in  Figure  A-6.  With 
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this  last  datum/  the  posterior  odds  of  an  invasion  have  gone 
over  the  50:50  chance  for  the  first  time.  In  fact,  the  odds 
are  now  3.9:1  in  favor  of  Hj,  or  almost  an  80%  likelihood  of 
an  invasion  occurring. 

Datum  #5; 


I 
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Prague,  June  18.  The  Soviet  commander  of  Warsaw  Pact 
forces  has  said  that  maneuvers  this  month  in  CzechoS' 
lovakia  would  involve  only  command  staffs  of  the  various 
'services  . . . The  Soviet  commander's  remarks  were 
apparently  intended  to  forestall  rumors  that  the  maneu- 
vers might  be  aimed  at  overturning  Czechoslovakia's 
liberal  regime. 

Until  this  point,  the  new  data  have  favored  H]^.  Datum  #5 
now  seems  to  indicate  that  the  implied  meaning  of  Datum  #4  is 
incorrect.  Inasmuch  as  there  is  no  way  of  determining  whether 
or  not  this  statement  is  only  propaganda,  the  analyst  might 
assess  a 60%  likelihood  of  this  statement  being  released  given 
there  is  going  to  be  an  invasion  and  80%  probability  if  there 
is  to  be  no  invasion;  a likelihood  ratio  equal  to  1:1.3  against 
Hi  giving  posterior  odd-i  of  3:1  (Figure  A-7) . 

Data  #6,  #7,  #8,  #9: 

The  next  four  pieces  of  data,  from  June  30  to  August  5, 
all  indicate  a decreasing  amount  of  tension  in  Czechoslovakia, 
favoring  Hi,  no  invasion.  Figure  A-8  represents  the  NY  Times 
articles  and  the  assessed  LR's,  and  their  impacts. 

Datum  #6: 

Prague,  June  30.  The  military  staff  exercises  of 
Warsaw  Pact  forces  that  created  anxiety  in  Czechos- 
lovakia for  about  a month  ended  today. 

P(D,Ih,)  .30  1 

LR  - 

® P(DglH2)  .80  2.7 


Datum  #7; 


f 


Prague,  July  13.  Some  of  the  Soviet  troops  still  in 
Czechoslovakia  two  weeks  after  the  end  of  the  joint 
Warsaw  Fact  maneuvers  started  for  home  today...  At  the 
same  time,  leaders  of  the  Soviet  Union  and  four  East 
European  allies  gathered  in  Warsaw  without  the  Czechos- 
lovaks to  discuss  Prague's  determination  to  make  the 
Communist  regime  more  democratic. 
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Figure  A-7 

DATUM  No.  B 

P(D7lHi)  .30  1 

^ P(D8lH2)  .60  2 

Datum  #8; 

Warsaw,  July  30.  Informed  sources  said  today  that  the 
Soviet  Army  had  concentrated  combat  units  in  Poland 
capable  of  armed  intervention  in  Czechoslovakia  and 
that  the  Soviet  troop  movements  in  the  last  three  days 
had  assumed  dimensions  in  excess  of  a simple  show  of 
force. 

Prague,  July  30.  Well-informed  Czechoslovak  sources 
believe  that  the  meeting  between  the  leaders  of  this 
country  and  those  of  the  Soviet  Union  has  served  to 
push  the  possibility  of  Soviet  military  intervention 
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further  into  the  background.  Their  belief  is  based  on 
authoritative  reports  that  the  first  day  of  negotia- 
tions at  Cierna,  Czechoslovakia/  yesterday  brought  no 
new  Soviet  demands  or  threats. 

PCDslHi)  .50  1 

LR  « ■ = ■»=  

P(D8lH2)  .55  1.1 


20:1 


10  1 

7;t 

e.i 


Datum  #9; 


Moscow,  Aug.  5.  The  specter  of  "counterrevolution"  in 
Czechoslovakia  that  evoked  cries  of  outrage  and  alarm 
from  the  Soviet  Union  seems  in  Moscow  to  have  disappeared 
abruptly  into  the  mists  of  unhistory.  Scornful  de- 
nunciations of  democratization,  allegations  of  anti- 
communist conspiracies  and  similar  ominous  accusations 
aimed  at  the  Prague  liberals  have  vanished  from  the 
Soviet  press.  Mass  propaganda  rallies  throughout  the 
country  to  stir  up  hostility  against  the  Czechoslovak 
liberation  program  have  ended. 

PCDglHi)  .30  1 

LRg  « ■■  » ■ — 

P(D9|H2)  .90  3 


Datum  #10; 

Moscow,  Aug.  9.  The  Soviet  Communist  Party  warned  its 
members  today  that  if  any  of  them  had  any  ideas  about 
liberalization  in  the  party  along  the  line  of  the 
recent  Czechoslovak  reforms,  they  had  best  forget  them. 
Pravda , the  party  organ,  printed  a long  article  de- 
fending  the  principle  of  "democratic  centralism"  and 
declaring  that  an  abandonment  of  tnis  system  of  unques- 
tioning obedience  to  decisions  from  the  top  would  lead 
to  a collapse  of  Communist  Party  rule. 

From  June  18  (Datum  #5)  to  August  5 (Datum  #9),  the 
differences  between  the  Soviet  Union  and  Czechoslovakia 
seemed  to  have  been  resolved,  and  general  relationships 
appeared  to  have  improved;  Datum  #10  is  the  first  real  piece 
of  evidence  that  those  differences  may  not  have  been  com- 
pletely resolved.  With  this  new  datum,  the  analyst  might 
assess  that  LR  ■■  .80/. 45  > 1.8/1.  Figure  A-9  shows  the 
effect  of  the  addition  of  this  new  information. 

Datum  #11; 

Moscow,  Aug.  16.  The  Soviet  Union  has  resumed  polemics 
against  the  Czechoslovak  press  after  a lapse  of  three 
weeks . 

The  Soviet  Union  has  now  renewed  direct  attacks  against 
Czechoslovakia.  Apparently  all  private  efforts  at  a recon- 
ciliation have  failed,  resulting  in  an  increasing  LR  « 

.85/. 25  = 3.4/1  (see  Figure  A-10) . 

Datum  #12 ; 

Moscow,  Aug.  18.  The  Soviet  Communist  Party  expressed 
apprehension  today  that  the  Czech  leadership  appeared 
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DATUM  No.  10 


to  be  losing  control  In  the  country.  "Subversive 
activities  by  antisocialist  forces"  have  resumed 
in  Prague,  Pravda,  the  Soviet  Party's  newspaper 
asserted.  It  suggested  that  the  Czechoslovak 
leaders  were  unable  to  cope  with  the  threatening 
situations. 

Soviet  criticism  of  Czech  leaders  is  now  greatly  in- 
creasing and  seems  to  imply  an  Invasion  is  imminent.  The 
analyst  assesses  an  80%  probability  that  these  remarks  would 
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Figure  A- 10 
DATUM  No.  11 

be  in  Pravda  if  an  invasion  were  planned,  and  only  a 20% 
probability  if  an  invasion  were  not  planned.  This  twelfth 
datum  is  a very  strong  one  with  an  LR  = 4.0/1  (see  Figure  A- 
11)  . 

Datum  #13: 

Prague,  Aug.  19.  The  press  agency  CTK  announced  that 
Czechoslovak  Army  divisions  will  exercise  Wednesday  and 
Thursday  in  Bohemia,  with  observers  from  other  Warsaw 
Pact  countries  looking  on. 

Datum  #13  is  the  last  piece  of  data  to  be  considered. 

It  is  obvious  now  that  there  is  an  intention  to  invade 
Czechoslovakia.  The  LR  » 1.0/. 12  = 8.3/1  and  raibes  the 
total  posterior  odds  to  33:1  or  about  a 97%  probability  in 
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favor  of  Hi,  the  Soviet  Union  is  planning  to  invade  Czechos- 
lovakia (see  Figure  A-12). 

Two  days  later  the  following  headlines  and  article  appeared 
in  the  NY  Times: 


CZECHOSLOVAKIA  INVADED  BY  RUSSIANS  AND  FOUR  OTHER 
WARSAW  PACT  FORCES 

Moscow,  Wed.,  Aug.  21.  Moscow  announced  this  morning 
that  troops  from  the  Soviet  Union  and  four  other  Com- 
munist countries  [on  Aug.  20]  had  invaded  Czechoslovakia 
at  the  request  of  the  "party-Government  leaders  of  the 
Czechoslovak  Socialist  Republic," 
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DATUM  No.  13 

In  this  appendix  the  events  leading  up  to  the  August  20, 

1968,  invasion  of  Czechoslovakia  have  been  studied.  The 
initial  assessment  of  prior  odds  1:20  against  the  invasion 
was  slowly  increased  until  May  24  when  it  was  announced  that 
the  armed  forces  of  the  Warsaw  Pact  would  hold  exercises  on 
Czech  territory.  At  that  time  the  odds  were  3.9:1  in  favor 
of  or  almost  an  80%  likelihood  of  an  invasion  occurring. 

After  that  point,  four  consecutive  sets  of  data  (#6-#9)  all 

indicated  a resolution  of  differences  and  a decreasing  I 

amount  of  tension  between  the  USSR  and  Czechoslovakia.  On 

August  5 (Datum  #9)  the  likelihood  of  an  invasion  dropped  to 

1:6  or  only  a 14%  chance.  But  on  August  9,  eleven  days  ■ 

before  the  actual  invasion,  the  Soviet  Union  resumed  press  y 

attacks  against  Czech  leadership.  Those  attacks  continued 
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vintil  August  19  (Datum  #13)  and  the  press  agency  CTK  announce- 
ment of  exercises  In  Czechoslovakia.  That  announcement 
raised  the  total  posterior  odds  to  33.1,  or  a 97%  probability 
of  invasion.  In  each  case,  Bayes*  Theorem  has  enabled  the 
analyst  to  revise  the  probabilities  as  new  information  was 
received.  It  permitted  him  to  substitute  quantity  for 
quality,  with  large  amounts  of  data  sufficing  in  situations 
where  the  Information  was  Incomplete  or  not  entirely  reli- 
able. Finally,  Bayes*  Theorem  allowed  the  analyst  to  com** 
bine  data  from  many  different  sources,  to  weigh  each  datum 
as  it  was  received,  and  to  visualize  the  effect  of  a new 
datum  graphically  on  a log  odds  chart. 
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APPENDIX  B 


BAYESIAN  INFERENCE  ABOUT  MANY-VALUED  UNCERTAINTIES 


In  Appendix  A,  "Inference  from  Evidence:  Bayes' 

Theorem,"  we  saw  how  "prior"  uncertainty  about  a two-valued 
event  (an  invasion  occurs  or  not)  can  be  updated  after  the 
receipt  of  evidence  (e.g.,  a certain  Intelligence  report), 
by  means  of  Bayes*  Theorem.  The  prior  probabilities  are 
combined  with  a "likelihood  ratio,"  which  measures  the 
diagnostic  value  of  the  evidence,  to  produce  "posterior" 
probabilities.  Essentially  the  same  prior-posterior  logic 
can  be  used  to  update  uncertainty  about  a many- valued  event, 
such  as  which  of  several  possibilities  (e.g.,  type  of  contract 
to  be  awarded)  or  what  value  on  a continuum  (e.g.,  size  of 
an  R&D  budget)  will  occur.  Instead  of  just  two  alternative 
hypotheses  (Hi  and  H2)  to  consider,  there  are  any  number  of 
hypotheses,  from  three  to  Infinity  (Hi,  H2,  H3,  ...,  and  so 
on,  or,  more  briefly,  Hi). 

As  with  two-valued  uncertainties,  the  key  inputs  are 
prior  probabilities  and  likelihoods  reflecting  the  diagnostic 
value  of  the  evidence.  Prior  probabilities,  p(Hi),  p(H2), 
P(H3),  and  the  like  are  written  as  p(Hi).  Likelihoods  asso- 
ciated with  a datum  of  evidence  are  written  as  p(D|Hi). 

In  the  special  two-value  case  discussed  in  Appendix  A,  the 
prior-posterior  calculation  Is  rather  simple.  The  posterior 
odds  are  calculated  as  the  product  of  the  prior  odds  and  the 
likelihood  ratio: 

P(Hi|D)  P(H,)  P(DlHi) 

■ JC  ; . (1) 

P(H2|D)  P(H2)  P(d|H2) 

When  there  are  more  than  two  hypotheses,  the  procedure  is 
somewhat  more  complicated. 

Bayes'  Theorem  tells  us  that  the  posterior  probability 
of  any  one  hypothesis  Hi  is  related  to  its  prior  probability 
and  its  likelihood,  given  the  evidence,  according  to  the 
following  formula: 


p:h.)p(d|h.) 

P(hJd)  -=  i i-  . (2) 

P(D) 

This  formula  is  repeated  for  each  of  the  possible  hypotheses, 
the  number  of  repetitions  indicated  by  the  largest  value  of  i. 
All  of  these  posterior  probabilities  must,  of  course,  sum  to  one 
on  the  assumption  that  one  of  the  hypotheses  is  true.  Therefore 
since  the  unconditional  probability  of  the  datum,  P (D) , 
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appears  in  the  denominator  of  the  posterior  probability 
expressions  for  all  H£,  equation  (2),  can  be  restated  as  a 
proportionality  indicated  by  the  symbol  «; 

P(H^|d)«P{H^)P(d|Hj^).  ' (3) 

Equation  (3)  states  that  the  posterior  probability  of  a 
hypothesis  is  proportional  to  Its  prior  probability  times 
the  likelihood  of  tho  datum  and  thereby  eliminates  the  need 
to  consider  F(D)  specifically.  Since  p(D)  serves  as  a 
constant  of  proportionality,  its  actual  value  can  be  deter- 
mined by  summing  the  right-hand  products  of  the  equations 
that  result  from  applying  Equation  (3)  for  each  hypothesis. 
That  is, 


P(D)  - X P(H^)P(D|Hj^)  . (4) 

i“l 

A simple  example  may  be  used  to  illustrate  the  application 
of  this  algebra. 

I 

j A Three- Valued  Example 

j Suppose  that  the  Commanding  Officer  of  an  aircraft 

' carrier  is  oona<arned  with  an  unidentified  plane  flying  on  a 

straight  course  to  his  ship.  It  may  be  American  or  non- 
American  and  either  friend  or  foe.  He  assigns  prior  probabilities 
of  0.5,  0.2,  and  0.3,  respectively.  When  the  plane  refui^es 
radio  acknowledgment,  the  Commanding  Officer  must  decide* 
whether  his  posterior  probabilities  would  justify  shooting 
the  plane  down.  His  reading  of  the  evidence  is:*  If^  the 
plane  were  American,  the  likelihood  of  refused  radio  communi- 
cation would  be  0.1;  if  other  friend,  0.3;  and  if  foe,  0.8.  i 

(Note  that  these  are  likelihoods  and  do  not  need  to  add  to 
one) . 

Figure  B-1  shows  the  calculation  of  the  posterior 
probabilities  (Column  5} , from  the  prior  probabilities  (Column 
2}  and  the  likelihoods  (Column  3).  The  joint  probabilities 
in  Column  4 are  the  products  of  Columns  2 and  3 for  each 
hypothesis,  and  the  sum  of  Column  4 is  P(D),  in  this  case, 

0.35.  The  posterior  probability  for  each  hypothesis.  Column  5, 
is  the  joint  probability  in  Column  4 divided  by  the  sum  of 
the  joint  probabilities,  P(D).  These  computations  are  all 
implied  by  equations  (2),  (3)  and  (4). 
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HVPOTHE*5l*!  PRIOR  . iifpi  JOINT  POSTERIOR 

HYPOTHESIS  probability  " v PROBABILITY  PROBABILITY 

P(H|)  P(H))P(D|H|)  P(H||D) 


The  same  worksheet  can  be  used  for  any  prior-posterior 
analysis#  regardless  of  how  many  hypotheses  there  are, 
provided  that,  for  each  hypothesis,  there  is  a prior  proba- 
bility and  a likelihood.  Each  hypothesis  will  correspond  to 
one  row  on  the  worksheet  as  in  Figure  B-1. 

Where  the  Likelihoods  Come  From 

It  makes  sense  to  use  this  kind  of  prior-posterior 
analysis  to  update  uncertainty  only  if  the  inputs  required, 
in  this  case,  prior  probabilities  and  likelihoods,  are  more 
conveniently  or  more  reliably  obtained  than  the  direct 
assessment  of  a posterior  distribution  would  be.  In  fact, 
since  likelihoods  are  often  quite  difficult  to  assess  sub- 
jectively, this  condition  will  not  necessarily  hold.  It  may 
be  quite  difficult  to  visualize  how  likely  it  is  that  you 
would  have  observed  the  evidence  that  you  have  observed, 
given  alternative  possible  hypotheses. 

However,  the  likelihoods  can  sometimes  be  determined 
quite  routinely,  notably  when  the  process  whereby  the  data 
generated  is  clear-cut  and  well-understood.  Suppose,  for 
example  that  the  "target"  uncertainty  is  the  proportion  of 
red  chips  in  a bag  of  red  and  white  chips,  and  suppose  that 
the  datum  observed  is  that  a red  chip  has  been  drawn  ran- 
domly from  the  bag.  In  this  case,  the  assessment  of  the 
likelihoods  presents  no  problem.  It  is  obvious  that  if  20% 
of  the  chips  are  red,  there  is  a 0.2  chance  that  a red  chip 
will  be  drawn;  if  40%  of  the  chips  are  red,  a 0.4  chance; 
and  so  on.  The  only  subjective  judgment  that  has  to  be 
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supplied,  therefore,  is  the  prior  probability  distribution. 
If  only  four  hypotheses  were  considered  (20%  or  40%  or  80% 
or  100%  of  the  chips  are  red) , and  each  of  these  were  given 
equal  probability,  Figure  B-2  shows  how  the  posterior  proba- 
bility distribution  would  be  computed.  The  last  column 
shows  that  the  posterior  probabilites  are  now  .08,  .17,  .33, 
and  .42,  respectively. 
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% RED  CHIPS 
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PROBABILITY 
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Figure  B-2 

PRIOR-POSTERIOR  ANALYSIS  WITH  OBVIOUS  LIKELIHOODS 


It  sometimes  happens  that  the  likelihoods  are  not 
obvious  but  oan  be  detexrmined  in  some  r/tanner,  such  as  by 
using  the  laws  of  probability  to  calculate  them  from  simple 
assessments  about  which  the  assessor  feels  relat.'ively  con- 
fident. Suppose  that  instead  of  one,  ten  chips  had  been 
drawn  from  the  bag  (and  replaced  after  each  drawing) , and 
six  of  them  were  red.  It  would  no  longer  be  obvious,  at 
least  to  a non-statistician,  what  the  likelihoods  would  be 
in  this  case.  Intuitively,  it  is  difficult  to  say  what  the 
probability  is  of  drawing  six  red  chips  out  of  ten,  if  the 
bag  they  are  drawn  from  has,  for  instance,  20%  red  chips. 
However,  a statistician  can  calculate  these  probabilities  if 
we  tell  him  the  likelihood  for  just  one  red  chip  (he  also 
has  to  know  that  we  put  each  chip  back  in  the  bag  after  it 
is  drawn,  and  that  the  red  and  white  chips  are  identical 
except  for  color) . The  results  of  his  calculations  are 
shown  in  Column  3 in  Figure  B-3,  and  the  prior-posterior 
analysis  proceeds  as  before.  Figure  B-3  shows  the  posterior 
probabilities  for  the  red  fraction  being  20%,  40%,  80%,  100% 
as  being  .03,  .54,  .43,  and  0,  respectively. 
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HYPOTHESIS 
% RED  CHIPS 

PRIOR 
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Figure  B-3 

PRIOR-POSTERIOR  ANALYSIS  WITH 
LIKELIHOODS  CALCULATED  FROM  SIMPLE  ASSESSMENTS 


The  situation  in  which  llKelihoods  can  be  determined 
easily  or  routinely  tends  to  occur  where  carefully  controlled 
samples  have  been  taken,  not  where  evidence  "just  turns  up." 

When  There  Are  Many  Possible  Values  of  the  Target  Uncertainty 

Although  real-world  quantities  are  very  rarely  strictly 
continuous  (even  the  GNF  roust  be  measured  to  the  nearest 
cent!),  their  possible  values  are  frequently  very  numerous, 
like  the  size  of  an  R&D  budget  or  the  maximum  range  of  a 
missile.  In  such  cases,  it  would  obviously  be  impractical  to 
list  all  possible  values  and  assign  prior  probabilities  and 
likelihoods  to  each.  Three  common  shortcuts  are  used  to 
avoid  this  difficulty. 

In  very  special  oases,  there  are  mathematical  shortcuts . 
Take,  for  example,  the  above  " chips- in-a-bag*^  example  I in 
which  six  out  of  ten  chips  prove  to  be  red.  Suppose  that 
instead  of  only  four  possible  values  for  the  proportion  of 
red  chips,  there  had  been  a large  number  of  values  (101  if 
there  were  100  chips  in  the  bag) . If  the  prior  distribution 
over  this  large  number  of  values  had  been  described  in  a 
special  mathematical  form,  called  a "beta"  function,  the 
required  posterior  distribution  could  have  been  obtained 


routinely  by  using  statistical  theory.  Since  this  condi- 
tion is  rarely  found  outside  certain  types  of  Scunple  survey 
situations,  we  shall  not  pursue  it  here. 

An  approximate  graphic  solution  can  often  be  obtained 
with  a little  practice.  Equation  ( 3 ) states  that  the  pos- 
terior probability  of  any  particular  value  is  proportional 
to  the  product  of  its  prior  probability  and  its  likelihood. 
If  the  probabilities  and  likelihoods  are  plotted  as  a func- 
tion of  the  variable,  a graph  of  the  posterior  distribution 
can  be  sketched  so  that  its  height  is  proportional  to  the 
heights  of  the  two  input  graphs. 

To  extend  an  earlier  example,  suppose  that  a red  chip 
has  been  drawn  from  a bag  with  possible  percentages  of  red 
chips  ranging  from  zero  to  100,  and  that  the  prior  proba- 
bilities are  as  indicated  in  Figure  B-4.  The  likelihood 
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Figure  B-4 

GRAPHIC  REPRESENTATION  OF 
PRIOR  PROBABILITY  AND  LIKELIHOOD 


In  fact,  the  posterior  distribution  is  a beta  distribution 
whose  two  parameters  are  those  of  the  prior  beta  distribu- 
tion with  the  size  of  the  sample  added  to  one  parameter  and 
the  number  of  "successes'*  added  to  the  other.  The  reader 
interested  in  getting  more  detailed  information  on  this 
special  case  and  others  like  it  is  referred  to  Howard 
Raiffa  and  Robert  Schlaifer,  Applied  Statistical  Decision 
Theory  (Cambridge,  HA;  Harvard  University  Press,  196l) . 


function  will  be  a straight,  upward- sloping  line  reflecting 
the  fact  that  the  probability  of  observing  a red  chip  is 
exactly  equal  to  the  fraction  of  red  chips  in  the  hag,  as  in 
Figure  B-4.  The  height  of  the  posterior  distribution  for  a 
given  percentage  of  red  chips  is  proportional  to  the  product 
of  the  heights  of  the  two  curves  for  the  same  percentage  in 
Figure  B-4.  A first  approximation  of  the  posterior  dis- 
tribution can  be  drawn  by  eye,  as  in  Figure  B-5. 


Figure  B-5 

VISUAL.  APPROXIMATION  OF 
POSTERIOR  PROBABILITY  FROM  CURVES  OF  FIGURE  B4 


The  third  shortcut  for  prior-posterior  analysis  on 
many-valued  uncertainties  is  probably  the  most  widely  appli- 
cable. Called  "grouping,"  this  shortcut  is  accomplished  by 
substituting  a few- valued  probability  distribution  for  a 
many-valued  probability  distribution.  The  procedure  essen- 
tially consists  of  div.!  ^ing  the  original  scale  of  values 
into  any  convenient  nun.^er  and  size  of  segments,  or  "groups, 
of  values  and  representing  each  group  by  a single  value. 
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original  curve  (Figure  B-6A)  are  concentrated  at  represen- 
tative values  in  the  substitute  few-valued  curve  (Figure  B- 
6B) , If  you  are  satisfied  that,  for  your  purposes,  the 
substitute  distribution  is  similar  to  your  real  prior  dis- 
tribution, you  can  proceed  as  if  it  were  your  prior  distri- 
bution. In  the  particular  example  of  Figure  B-6  where  there 
are  only  five  groups,  there  would  only  be  five  rows  in  the 
prior-posterior  worksheet,  instead  of  the  very  large  numbers 
of  rows  that  would  have  been  necessary  had  the  ungrouped 
prior  distribution  been  used. 

Certain  methods  of  effecting  this  grouping  are  moderately 
convenient  and  reliable.  One  of  them  is  called  the  bracket 
median  method . To  use  this  method,  you  draw  the  prior 
probability  distribution  in  the  form  of  a cumulative  curve, 
pick  up  the  .05,  0.15,  0.25,  through  to  0.95  fractlles,  and 
assign  0.1  probability  to  each.  Figure  B-7  Illustrates  the 


A.  Cumulative  Curve  of  Prior  Probability  Diitribution 


Figure  B-7 

GROUPING  BY  BRACKET  MEDIAN  METHOD 
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bracket  median  method  in  the  context  of  the  "red  fraction" 
example  used  before.  The  original  prior  distribution  shown 
in  cumulative  form  in  Figure  B-7A  is  reduced  to  a ten-point 
probability  distribution  with  equal  probabilities  in  Figure 
B7-B.  Figure  B-8  shows  the  worksheet  calculations  for  a 
prior-posterior  analysis  using  the  grouped  prior  distribu- 
tion derived  in  Figure  B-7. 
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Figure  B-8 

PRIOR-POSTERIOR  ANALYSIS 
USING  GROUPED  PRIOR  PROBABILITY  DISTRIBUTION 


Note  that  the  output  of  this  analysis  is  itself  grouped; 
that  is,  the  posterior  distribution  has  ten  possible  values 
in  this  case.  The  possible  values  are  the  same  as  those  in 
the  grouped  prior  distribution,  but  the  probabilities  are 
different.  A many-valued  cumulative  curve  can  be  plotted  by 
what,  in  effect,  is  the  reverse  of  grouping.  The  cximulative 
distribution  for  the  ten-value  posterior  distribution  is 
given  in  Column  6 of  Figure  B-8.  These  values  can  be  plotted 
and  a smooth  curve  drawn  through  them,  as  in  Figure  B-9.  If 
all  that  is  needed  from  the  posterior  distribution  ie  some 
summary  measure  like  a mean  or  a variance,  this  measure  can 
be  calculated  directly  from  the  grouped  posterior. 


Figure  B-Q 

DEGROUPING  THE  POSTERIOR  DISTRIBUTION 
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APPENDIX  C 


A SCORING  RULE  FOR  PROBABILITY  ASSESSMENT 


We  have  argued  that  probability  assessments  should  be 
coherent  and  that  they  should  obey  the  rules  of  probability 
theory.  But  it  is  not  enough  that  they  obey  the  rules;  they 
should  also  be  related  to  reality.  A decision  maker  would 
soon  lose  in  probability  assessments  if  it  turned  out 

that  the  ev>.r:;'t::  that  occurred  were  associated  with  low 
probablllt)  ■ "I  those  that  did  not  occur  were  associated 
with  high  prvi^f^bilitios. 

The  ideal  in  probability  assessments  is  clairvoyance, 
that  is,  the  assignment  of  a probability  of  100%  to  the 
events  that  turn  out  to  be  true,  and  0%  to  the  events  that 
are  not  true.  But  clairvoyance  is,  of  course,  not  possible. 
Since  human  beings  are  unfortunately  not  blessed  with  powers 
of  omniscience  and  future  events  are  only  partially  acces- 
sible to  their  foresight,  the  ideal  standard  of  clairvoyance 
is  not  obtainable. 

Proper  Scoring  Rules 

Clairvoyance  is,  however,  a useful  standard  against 
which  actual  probability  assessments  can  be  evaluated.  They 
can  be  measured  against  this  standard  for  accuracy  (or 
error)  by  scoring  rules  developed  for  that  purpose.  Since 
much  of  the  development  of  proper  scoring  rules  has  occurred 
in  the  area  of  weather  forecasting,  where  scoring  rules  are 
used  to  evaluate  precipitation  probability  forecasts,  we 
shall  use  the  following  example  of  precipitation  proba- 
bilities to  explain  the  concept  of  a proper  scoring  rule.  A 
weather  forecaster  assesses  a precipitation  probability 
between  0%  and  100%  and  he  later  measures  whether  or  not  it 
rained.  He  decides  that  it  has  rained  if  his  rain  gauge 
gathers  more  than  .01  inch  of  moisture;  otherwise,  it  has 
not  rained.  Given  his  probability  forecast  about  rain,  and 
later  information  about  whether  it  does  or  does  not  rain, 
how  good  is  his  forecast? 

Consider  first  an  intuitively  appealing  but  improper 
scoring  rule,  called  a linear  scoring  rule.  The  first  step 
is  to  measure  the  actual  distance,  in  probability  units, 
between  a clairvoyant  assessment  and  the  actual  assessment. 
For  example,  if  a 60%  precipitation  probability  is  assessed, 
and  is  followed  by  rain,  then  the  error  score  would  be  40 
points,  the  distance  between  100%  and  60%.  If,  on  the  other 
hand,  a precipitation  probability  of  60%  is  assessed  but  is 
not  followed  by  rain,  then  the  error  score  would  be  60 
points,  the  difference  between  the  clairvoyant  0%  and  the 
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assessed  60%.  Figure  C-]  displays  this  scoring  rule.  The 
error  score  Is  plotted  on  the  vertical  axis  as  a function  of 
the  percentage  probability  assigned  to  the  true  event  which 
is  displayed  on  the  horizontal  axis.  Accordingly r if  100% 
is  assigned  to  the  true  event,  there  is  an  error  of  0;  if 
50%  is  assigned  to  the  true  event,  the  error  score  is  50 
points;  and  if  0%  is  assigned  to  the  true  event,  the  error 
score  is  100  points. 


LU 

ae 


Figure  C-1 

LINEAR  SCORING  RULE 
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In  order  to  understand  why  this  apparently  reasonable 
scoring  rule  in  Figure  C-1  is  inappropriate,  consider  the 
following  example.  After  careful  consideration  of  all 
relevant  information,  an  expert  forecaster  assesses  a 70% 
precipitation  probability.  First,  suppose  that  he  tells  the 
truth  when  reporting  his  assessment.  In  other  words,  he 
reports  a precipitation  probability  of  70%.  What  is  his 
expected  score  for  telling  the  truth?  If  it  rains,  he  will 
receive  an  error  score  of  30  points  since  he  has  assessed  a 
70%  probability  that  it  will  rain.  If  it  does  not  rain,  he 
will  receive  an  error  score  of  70  points  since  he  has  assessed 
a 30%  probability  of  that  event.  Therefore,  his  expected 
error  score  is  equal  to  70%  of  30  points  plus  30%  of  70 
points,  which  is  21  + 21,  or  42  points. 

Mow  suppose  that  our  forecaster  decides  not  to  tell  the 
truth  when  reporting  his  assessment.  Specifically , suppose 
that  he  reports  a precipitation  probability  of  100%.  In 
this  case,  what  is  his  expected  score?  If  it  rains,  he  will 
receive  an  error  score  of  0 points.  If  it  does  not  rain, 
he  will  receive  an  error  score  of  100  points.  His  expected 
score  is  equal  to  70%  of  0 points  plus  30%  of  100  points, 
which  is  0 plus  30,  or  30  points.  Since  the  error  of  30 
points  is  a much  smaller  penalty  score  than  is  42  points, 
our  weather  forecaster  is  well  advised  not  to  tell  the  truth 
but  rather  to  report  a 100%  precipitation  probability. 

Clearly,  this  is  an  unsatisfactory  state  of  affairs. 

A scoring  rule,  to  be  useful  in  evaluating  probability 
assessments,  should  have  the  property  that  the  probability' 
assessor  not  only  need  not  "play  games"  in  order  to  do  well, 
but  also  is  rewarded  for  telling  the  truth.  A class  of 
scoring  rules,  called  proper  scoring  rules,  has  been  developed 
whereby  the  probability  assessor  can  mlnTmize  his  expected 
penalty  score  only  by  reporting  his  true  assessed  proba- 
bility. 

The  Brier  score,  which  is  used  for  weather  forecasting, 
is  an  example  of  a proper  scoring  rule.  In  the  two-event 
case,  of  which  the  precipitation  problem  is  an  example,  the 
Brier  rule  assigns  an  error  score  which  is  equal  to  .01 
times  the  square  of  the  distance  of  the  assessed  probability 
from  clairvoyance.  It  is  calculated  by  finding  the  true 
event  and  then  squaring  the  distance  of  the  probability 
assessed  for  that  event  from  100%  or  0%,  depending  on  whether 
the  event  confirms  or  refutes  the  assessment.  For  example, 
consider  an  assessed  precipitation  probability  of  60%. 

If  it  rains,  the  assessment  is  assigned  an  error  score  of 
.01  X (100-60)2,  or  16  points.  If  it  does  not  rain,  the 
assessment  is  assigned  an  error  score  of  .01  x (60-0)2,  or 
36  points.  The  proper  Brier  score  is  thus  a very  simple 
modification  of  the  improper  linear  scoring  rule. 
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Figure  C-2  displays  the  Brier  rule.  The  error  score  is 
shown  on  the  vertical  axis  as  a function  of  the  percentage 
probability  assigned  to  the  true  event,  which  is  shown  on 
the  horizontal  axis.  Now  apply  the  Brier  score  to  the 
weather  forecaster  whose  assessed  precipitation  proba- 
bility is  70%.  Suppose  he  reports  this  assessment  as  it 
stands.  If  it  rains,  he  will  receive  an  error  score  of 
9;  if  it  does  not  rain,  he  will  receive  an  error  score  of 
49.  His  expected  score,  then,  la  70%  of  9 (6.3)  plus  30%  of 
49  (14.7),  or  21  points.  Next,  consider  the  case  in  which 
he  "plays  games"  and  reports  a precipitation  probability  of 
100%.  If  it  rains,  he  will  receive  an  error  score  of  0;  if 


PROBABILITY  (%)  ASSESSED  FOR  TRUE  STATEMENT 

Figure  C-2 

BRIER  SCORING  RULE 


it  does  not  rain,  he  will  receive  an  error  score  of  100 
points.  Against  the  standard  of  a truthful  report  of  his 
true  assessment  of  the  probabilities,  his  expected  error 
score  is  70%  of  0 (0)  plus  30%  of  100  (30),  or  30  points. 
Since  an  error  score  of  30  is  substantially  greater  than  a 
score  of  21,  the  forecaster  is  well  advised  to  report  truly 
his  best  assessment  of  his  probability  of  precipitation. 
Indeed,  it  can  be  shown  mathematically  that  a weather  fore- 
caster can  earn  the  highest  expected  Brier  score  only  by 
reporting  his  true  precipitation  probability. 

Sorting  Errors  Versus  Labeling  Errors 

Even  the  most  expert  forecaster,  however  truthful  his 
reports  of  assessed  probability  are,  will  be  wrong  some  of 
the  time.  His  assessments  have  no  bearing  on  the  outcomes 
which  may  confirm  or  refute  them.  However,  it  is  possible 
to  help  the  forecaster  make  statements  of  assessed  proba- 
bility which  have  the  same  probability  of  being  confirmed  as 
he  assesses  the  probabilities  of  the  outcomes.  In  short, 

70%  of  all  of  his  statements  assessing  a probability  of  70% 
ought  to  be  confirmed. 

If  not,  the  forecaster  is  making  one  of  two  kinds  of 
errors.  These  are  a sorting  error  or  a labeling  error.  We 
may  think  of  assessing  probabilities  as  a two-step  procedure. 
First,  we  consider  whether  a certain  event  will  happen  or 
not;  then,  to  what  degree  we  have  oonfidenoe  in  that  deter- 
mination. (The  manner  in  which  we  computed  a proper  scoring 
rule  depends  on  a comparable  procedure,  first,  what  happens 
in  the  outcome,  then  a computation  of  a score  based  on  the 
differences  between  tue  probability  and  the  certainties.)  A 
sorting  error,  then,  is  deciding  wrongly  that  an  event  will 
or  will  not  happen;  a labeling  error,  an  improper  degree  of 
confidence  in  that  decision. 

The  following  example  illustrates  these  two  error  com- 
ponents. Intelligence  analysts  in  a Washington  intelligence 
agency  made  weekly  forecasts  of  many  different  kinds  of 
events  such  as  whether  a military  coup  would  occur  within  a 
particular  time  interval,  whether  a reconnaissance  plane 
would  be  shot  down,  or  whether  an  arms  shipment  would  be 
made  to  a particular  country  within  a specified  time  inter- 
val. In  each  case,  it  was  possible  to  determine  some  time 
after  the  forecast  whether  or  not  the  event  in  question 
occurred,  that  is,  whether  or  not  the  statement  for  which 
the  probability  was  assessed  turned  out  tc  be  true. 

The  probability  assessments  were  evaluated  in  the 
following  manner.  First,  the  assessments  wore  categorized 
into  common  probabilities.  Thus,  till  assessments  of  70% 
were  placed  into  one  category,  assessments  of  40%  were 
placed  into  another  category,  assessments  of  10%  into  yet 
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another  category,  and  so  for  all  different  probability 
assessments  that  the  analysts  used.  The  goal  of  this  analy- 
sis was  to  calculate  the  percentage  of  true  statements, 
known  as  the  hit  rate,  associated  with  each  category.  In  an 
effort  to  obtain  dependable  hit  rates,  adjacent  categories 
ware  combined  so  that  the  combined  categories  contained 
approximately  100  different  asseesments.  Then  the  percen- 
tage of  true  statements,  or  the  hit  rate,  was  calculated 
within  each  category  by  dividing  the  number  of  true  state- 
ments by  the  total  number  of  statements  in  that  category. 
Figure  C-3  displays  the  results  of  this  analysis.  The 


HIT  RATE 


vertical  axis  refers  to  the  average  assessed  probability  for 
each  category  and  the  horizontal  axis  refers  to  the  corre- 
sponding hit  rate  in  the  category.  These  results  seem  to 
indicate  good  performance  on  the  part  of  the  analysts.  As 
the  average  assessed  probability  increased,  there  was  an 
accompanying  increase  in  the  percentage  of  correct  state- 
ments. An  exact  measure  of  performance  can  be  obtained  by 
using  the  Brier  rule  to  calculate  the  average  error  score 
for  all  assessments  that  contributed  to  each  data  point  in 
Figure  C-3. 

Since  the  data  points  do  not  all  fall  on  the  line 
indicating  a perfect  correlation  between  the  assessed  proba- 
bilities and  the  percentage  of  true  statements,  one  or  both 
kinds  of  errors  are  affecting  the  hit  rate.  Let  us  take  an 
exeuaple  that  measures  the  contribution  each  makes  to  the 
error  score. 

Consider  a hypothetical  data  point  at  which  the  average 
assessed  probability  is  equal  to  70%  and  the  hit  rate  is 
equal  to  60%.  The  sorting  error  is  a measure  of  the  degree 
to  which  the  analyst  failed  according  to  the  standard  of 
clairvoyance  to  sort  perfectly  all  true  statements  from  all 
false  statements.  Specifically,  the  sorting  component  of 
the  average  error  for  a data  point  is  equal  to  (P) (1-P)/100, 
where  P is  equal  to  the  hit  rate.  In  this  case,  the  average 
error  attributable  to  sorting  is  (60  x 40)/100,  or  24. 

Figure  C-4  illustrates  the  manner  in  which  the  sorting  com- 
ponent of  the  error  changes  with  P.  The  sorting  component 
is  at  0 for  perfect  sorting,  when  P is  equal  to  either  0%  or 
100%.  It  is  at  a meocimum,  25  points,  when  the  hit  rate  is 
equal  to  50% o Intuitively,  the  sorting  error  can  be  thought 
of  as  a measure  of  the  degree  to  which  the  probability 
assessor  is  knowledgeable  about  the  subject  matter.  To  the 
degree  that  he  is  knowledgeable  and  that  his  probability 
assessments  serve  to  sort  the  statements  into  almost  all 
true  statementa  zmd  almost  all  false  statements,  the  sorting 
component  of  his  error  score  will  be  low.  If  he  is  not  very 
knowledgeable,  and  his  probability  assessments  do  not  dis- 
criminate very  well  between  true  and  false  statfuaents,  then 
the  sorting  component  of  his  error  score  will  be  high. 

The  second  component  of  the  error  score  results  from 
errors  in  the  labeling  process.  Having  decided  which  event 
is  likely  to  occur,  that  is,  having  decided  how  much  he 
knows  about  the  likelihood  of  a certain  outcome,  he  then 
determines  his  degree  of  confidence  in  his  decision.  Ideally, 
the  label  should  be  equal  to  the  hit  rate.  For  exeunple,  if 
a weather  forecaster  is  doing  a good  job,  then  he  should  use 
the  probability  label  in  such  a manner  that  it  will  turn  out 
to  rain  on  about  70%  of  the  days  for  which  he  issues  a 70% 
precipitation  probability;  and  it  should  turn  out  to  rain  on 
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about  20%  of  the  days  for  which  he  has  assessed  ®20%  proba- 
bility. In  general,  the  assessed  probabilities  should  be 
equal  to  the  corresponding  hit  rates.  To  the  ^ ^ 

the  data  points  are  either  above  or  below  the  identity  line, 
we  infer  that  the  probability  assessor  has  erred  in  the 
process  of  labeling.  In  terms  of  Fig'll®  C- 3,  an  error-free 
labeling  process  would  imply  that  the  data  points  would  lie 
along  the  45-degree  line. 


ERROR  IN  PROBABILITY  ASSESSMENT  (%) 


Figure  C-4 

RELATIVE  CONTRIBUTION  OF 
COMPONENT  ERRORS  TO  ERROR  SCORE 

The  exact  contribution  of  the  error  in  labeling  to  the 
error  score  is  equal  to  the  square  of  the  difference  between 
the  assessed  probability  and  the  hit  rate,  divided  by  100. 
Accordingly,  in  our  ex£unple  in  which  the  assessed  probability 


is  70%  and  the  hit  rate  is  60%,  the  labeling  component  of 
the  error  score  is  equal  to  the  square  of  the  difference 
between  70  and  60,  all  divided  by  100:  it  is  only  1. 

Figure  0-4  shows  the  manner  in  which  the  labeling  component 
contributes  to  the  error  score  as  the  distance  between  the 
assessed  probability  and  the  hit  rate  increases.  The  labeling 
component  contributes  0 points  when  the  assessed  probability 
is  equal  to  the  iiit  rate,  and  it  contributes  100  points  when 
there  are  100  percentage  points  between  the  two. 

These  calculations  for  our  example  imply  that  the 
average  error  for  all  assessments  is  equal  to  25  points;  24 
are  attributable  to  sorting  error,  and  1 ia  attributable  to 
labeling  error.  Thus,  for  that  data  point,  96%  (.24/. 25)  of 
the  error  can  be  attributed  to  improper  sorting  wheread  only 
4%  (.01/  .25)  can  be  attributed  to  mislabeling.  Thus,  this 
particular  error  in  labeling  is  relatively  unimportant  as 
compared  with  the  inability  to  sort  appropriately  or  dis- 
criminate between  true  and  false  events.  The  relative 
degree  of  importance  will,  of  course,  vary  with  the  data 
point  analyzed.  Data  points  toward  the  extreme,  where  the 
hit  rate  is  near  0%  or  100%,  will  have  relatively  small 
amounts  of  sorting  error  where  data  points  in  the  center 
will  have  large  amounts.  The  degree  of  labeling  error 
depends  only  upon  the  vertical  distance  of  a data  point  from 
a straight  line.  The  degree  of  labeling  error  reflected  in 
Figure  C-3  is,  for  the  most  part,  relatively  small  because 
few  data  points  fall  very  far  from  the  straight  line. 

Notice  that  a large  labeling  error,  say,  above  50%, 
will  generally  occur  only  when  a probability  assessor  is 
being  deliberately  misleading.  That  is,  it  will  occur  when 
a probability  assessor  assigns  high  probabilities  to  state- 
ments that  usually  turn  out  to  be  false  and  low  probabilities 
to  statements  that  usually  turn  out  to  be  true.  Even  if  the 
probability  assessor  is  uncommonly  benighted,  this  labeling 
error  can  be  extreme  only  when  the  hit  rate  is  extreme  and 
thus  permits  a substantial  degree  of  error. 

Sorting  errors  can  be  reduced  primarily  by  improving 
the  analytic  process,  either  by  making  more  information 
available  or  by  providing  a better  means  for  processing  the 
available  information.  Labeling  errors,  on  the  other  hand, 
can  be  reduced  by  improving  the  process  by  which  the  proba- 
bility assessor  assigns  numbers  to  reflect  his  degree  of 
knowledge.  A better  assignment  of  numbers  requires  an  in- 
tuitive appreciation  for  the  meaning  of  a quantitative 
probability  scale. 

A computer-based  scoring  rule  training  procedure  has 
been  developed  to  calibrate  and  improve  the  accuracy  of 
probability  assesscrs.  In  application,  the  computer  poses  a 
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series  of  multiple-choice  questions  to  the  trainee.  The 
trainee  is  required  to  indicate  the  correct  answer  along 
with  a probabilistic  assessment  of  his  degree  of  certainty 
about  the  designated  answer.  Automatic  feedback  as  to  the 
accuracy  of  the  trainee's  response  is  provided , and  the 
computer  maintains  a running  calculation  of  the  implication 
of  cumulative  response  accuracy  and  uncertainty  levels  to 
determine  and  display  the  degree  of  sorting  error  and  labeling 
error  in  the  trainee's  performance.  In  experimental  situa- 
tions « use  of  this  computer-based  procedure  has  yielded 
ga.-’.ns  in  assessment  accuracy  of  from  10  to  40%. 
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APPENDIX  D 


PROBLEMS  IN  HIERARCHICAL  INFERENCE  AND  A CASE  STUDY 


In  this  appendix  we  first  discuss  the  general  princi- 
ples to  be  followed  in  naking  the  calculations  whose  results 
appear  on  the  inductive  structure  of  a hierarchical  inference 
problem.  Next,  we  mention  an  error  that  is  easily  made  when 
modifying  the  calculations  upon  receipt  of  a new  datum. 
Finally,  we  explain  in  some  detail  a case  study  in  hier- 
archical inference. 

Principles 

In  the  process  of  proceeding  inductively  up  through  the 
hierarchical  structure  for  the  nuclear  weapons  development 
program  discussed  in  Chapter  4,  the  analyst  was  confronted 
with  one  of  two  situations  at  each  branch  point.  The  first 
situation  occurs  whenever  it  is  necessary  to  determine  the 
likelihood  of  two  or  more  data  for  a given  indicator  or 
activity,  that  is,  whenever  two  or  more  data  branches  merge 
on  the  diagram  of  the  hierarchical  structure.  A diagram 
illustrating  this  situation  and  the  general  equation  (1) 
used  in  the  solution  is  shown  below: 


PlD|ey  - PlDi  02 Dp|ej^l  - ^ P(D,  | ej^  ) { 1) 

/-I 

FOR  k = 1.  2 n 


This  equation  says  that  the  probability  of  all  the  data 
given  each  state  of  the  variable  is  equal  to  the  product  of 
the  probability  for  each  datum  for  a given  state,  provided 
that  the  data  are  independent.  This  equation  is  an  appli- 
cation of  the  multiplication  rule  given  in  the  section 
"Rules  for  Combining  Probabilities,"  in  Chapter  4. 

For  example,  this  situation  occurred  in  the  nuclear 
development  problem  when  the  analyst  computed  likelihoods 
for  the  combination  of  datum  5 (D5)  and  datum  6 (Dg) , on  the 


assur.t  tioiis , first,  that  indicator  2 has  occurred  (I2)  / 
thei:  that  it  has  not  occurred  (I2)  . (Figures  4-35  and  4-40 
refer.)  Using  equation  1,  he  proceeded  as  follows: 

F{Dil2)  = P(D5ll2)  r(D6!l2) 

= (0.99)  (0.75)  = a. 74 

P(Dll2)  = r(05!i2)  P(D6|i2) 

= (0.50)  (0.70)  = 0.35  . 

D refers,  in  this  case,  to  the  combination  of  D5  and  Dg. 
These  computed  likelihoods,  0.74  and  0.35,  can  be  seen  in 
their  appropriate  places  on  Figure  D-l. 


The  second  situation  occurs  whenever  there  is  no  merging 
of  branches  and  it  is  necessary  to  determine  the  likelihood 
of  the  data  given  the  activity  or'  indicator  at  the  next 
higher  level.  A diagram  of  this  situation  along  with  the 
general  equation  (2)  used  in  .the  solution  is  shown  below; 


TV^is  equation,  which  applies  the  addition  rule  for 
probabi\ities  given  in  Chapter  4,  says  that  the  probability 

of  the  'jjcu:;r.ing  for  a given  state  of  the  upper-level 

\ 

\ 

\ 
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event  is  found  by  multiplying  each  probability  in  the  lower 
box  by  each  probability  in  the  first  column  of  the  upper  box 
(i.e.,  the  left-most  n\imber  in  the  row  by  the  top  number  in 
the  column,  the  second  from  left  in  the  row  by  the  second  from 
top  in  the  column  etc . ) and  by  summing  those  products , and 
then  by  repeating  this  procedure  for  each  remaining  column. 


For  example,  this  situation  occurred  in  the  current 
problem  just  after  the  analyst  computed  the  likelihoods  given 
on  the  previous  page,  at  the  point  where  he  wanted  to  calcu- 
late the  likelihoods  for  the  combined  effects  of  D5  and  Dg 
with  activity  1 as  the  conditioning  event  rather  than,  as 
previously,  indicator  2 as  the  conditioning  event.  Using  the 
previously-computed  likelihoods  along  with  the  assessed 
probabilities  relating  activity  1 to  indicator  2,  he  applied 
equation  2 as  follows: 


P(d1Aj_)  - P(Dll2)  + P(d1I2)  P(i2lAjL) 

» (0.74)  (0.80)  + (0.35)  (0.20) 

» 0.59  + 0.07  » 0.66 

P(D|Aj^)  - P(Dll2)  P(l2lAj^)  + P(D|i2)  ^(121^1^ 
« (0.74)  (0.60)  + (0.35)  (0.40) 

* 0.45  + 0.14  » 0.59  . 


These  likelihoods,  0.66  and  0.59,  are  also  shown  on  Figure 
D-1. 


(At  this  point,  the  reader  is  encouraged  to  examine  both 
the  deductive  and  inductive  structures,  to  determine  which 
rule  applies  to  the  generation  of  the  remaining  row  vectors, 
and  to  perform  the  indicated  calculations,  in  order  to  rein- 
force understanding  of  the  computational  procedure.) 

Suppose  the  analyst  is  now  given  a new  datum,  Dg, 
equally  as  diagnostic  about  increased  scientific  activity  as 
datum  7,  decrease  in  publications  on  high-explosives  research. 
If  the  analyst  is  careless  or  does  not  understand  how  to 
proceed,  he  may  attempt  to  incorporate  this  datum  in  the 
following  way:  To  accommodate  the  new  datum,  he  adds  to  the 
diagram  of  Figure  D-1  a fifth  branch  from  the  top  box. 

Since  the  impact  of  the  new  datum  is  the  same  as  datum  7,  he 
simply  duplicates  on  the  new  branch  all  the  boxes  and  their 
associated  numbers  shown  in  the  extreme  right  branch  of 
Figure  D-1,  Recall  that  the  likelihood  ratio  of  4.48  shown 
in  the  top  box  is  calculated  by  multiplying  together  the 
four  likelihood  ratios  on  the  left  sides  of  the  four  boxes 
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Figure  D-1 

INDUCTIVE  HIERARCHICAL  STRUCTURE 
FOR  NUCLEAR  WEAPONS  DEVELOPMENT  PROGRAM 


f on  the  next  level  below.  Thus,  to  take  account  of  the  new 

datum,  another  2.56  must  be  Included  in  the  product.  This 
same  line  of  reasoning  applies,  of  course,  to  the  calcula-* 
tion  of  a new  likelihood  ratio  for  no  nuclear  capability, 
the  right  figure  in  the  top  box.  So,  the  analyst  combines 
the  likelihood  for  the  new  datum  with  the  previous  likeli- 
hood at  the  top  of  Figure  D-1; 
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But  is  this  the  correct  inference,  given  the  additional 
datum? 

It  is  easy  to  show  that  this  is  an  incorrect  inference. 
Assume,  for  the  moment,  that  some  datum  which  tells  us  for 
certain  that  increased  scientific  activity  is  in  progress 
has  been  obtained,  that  is,  that  I3  has  definitely 
occurred.  Then  the  likelihood  of  this  datum,  given  that  a 
high-explosive  R&D  program  either  is  or  is  not  in  progress, 
is  as  shown  in  the  top  row  of  the  middle  box  in  the  right 
branch  of  Figure  D-2.  It  is  0.8  if  the  h-e  R&D  program  is 
underway , 0.4  otherwise . 


Figure  D-2 

DEDUCTIVE  HIERARCHICAL  STRUCTURE 
FOR  NUCLEAR  WEAPONS  DEVELOPMENT  PROGRAM 
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Carrying  out  calculations  as  before,  but  with  this 
modification  of  the  middle  box,  gives  the  new  values  shown 
in  Figure  D-3.  Note  that  the  likelihood  at  the  top  is  3.62 
to  1,  based  on  the  assvunption  of  perfect  information  for  the 
right  branch.  But  in  adding  a new  piece  of  imperfect 
information,  the  analyst  arrived  at  a likelihood  of  81  to 
1.  Clearly,  additional  imperfect  information  in  one  branch 
should  not  make  us  more  certain  than  when  perfect  informa- 
tion is  received  for  that  same  branch.  So,  the  procedure  he 
adopted  in  arriving  at  the  figure  of  3.81  to  1 must  be 
incorrect.  His  mistake  was  to  add  an  entire  fifth  branch 
when  the  new  datum  was  received.  This  mistake  is  equivalent 
to  an  assumption  that  the  seventh  and  eighth  data  were 
independent  with  respect  to  the  hypotheses  and  H2*  In 
fact,  they  are  independent  only  with  respect  to  the  indi- 
cator 1 3.  Thus,  the  new  datum  must  be  shown  entering  the 
diagram  just  below  I3. 


Nuo  Hue 
Cap  Cip 


Figure  D-4  shows  the  inference  structure  which  correctly 
relates  D9  to  the  hierarchy,  and  the  correct  likelihood  of  D 
given  H. 

Figure  D-4  demonstrates  the  following  rule;  new 
information  about  indicators,  activities,  or  hypotheses  must 
be  aggregated  with  the  existing  information  at  the  point  at 
which  they  enter  the  hierarchical  structure. 

The  Generic  Hierarchical  Structure 

We  would  like  to  present  now  a complete  case  study  of 
hierarchical  inference  as  a technique  for  the  solution  of 
complex  inference  problems,  but  first  it  will  be  useful  to 
introduce  notations 1 conventions  that  will  apply  throughout 
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Figure  D-4 

INFERENCE  STRUCTURE  GIVEN  DATUM  0^12 


the  remainder  of  this  appendix.  Figure  D-S  shows  the  struc- 
ture formulated  for  a hypothetical  problem,  which  contains 
four  levels. 

The  first  level  contains  the  exhaustive  set  of  mutually 
exclusive  hypotheses,  Hi,  required  to  describe  the  central 
issue. 

The  second  level  contains  the  data,  D^,  and  the  activ- 
ities, A^,  which  impact  directly  on  the  hypotheses.  These 
elements  are  numbered  with  a single  superscript,  in  consecu- 
tive order  from  left  to  right  on  the  diagram.  Each  category 
of  activity,  A^,  may  be  described  by  a number  of  mutually 

exclusive  and  exhaustive  states  A^  which  are  diagnostic  of 

J 2 

the  hypotheses.  For  example,  activity  A might  have  three 


2 2 2 

states,  which  would  be  Indicated  A.,  A2>  and  A^. 


1)1  a2  d3 
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Figure  D-B 

ILLUSTRATIVE  HIERARCHICAL  STRUCTURL 


The  third  level  of  the  hierarchy  contains  those  data 
and  indicators  which  impact  on  the  hypotheses  through  activ- 
ities in  the  second  level.  The  first  superscript  is  their 
consecutive  order  from  left  to  right  on  the  diagram.  Each 
indicator  may  consist  of  a number  of  mutually  exclusive  and 
exhaustive  states  which  are  diagnostic  of  the  related  activity. 

23 

These  states  are  indicated  by  consecutive  subscripts  (I.  , 

O a . * 


The  fourth  level  of  the  hierarchy  contains  those  data 
which  impact  on  the  hypotheses  through  indicators  in  the 
third  level  and  activities  in  the  second  level.  Each  datum 
is  identified  by  the  superscript  of  the  indicators  to  which 
it  is  related,  with  the  third  superscript  being  the  con- 
secutive order  of  occurrence,  in  relation  to  that  indicator, 
from  left  to  right  in  the  diagram. 

Complete  Case  Study 

Now  we  turn  to  the  case  study.  We  shall  use  the 
methodology  described  in  Chapter  4,  with  the  technical 
refinemeni's  described  earlier  in  this  appendix. 

Imagine  an  analyst  who  has  been  given  the  assignment  of 
determining  if  re;,  'jarch  which  could  lead  to  the  capability 
of  detecting  a moving  submarine  by  infrared  or  radar  obser- 
vation of  the  surface  effects  is  being  conducted  by  Country  B. 
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This  analysis  would  be  important  because  the  development 
of  such  a capability  would  allow  the  use  of  high~altitude 
aircraft  in  detecting  and  tracking  submarines  and  would  elimi- 
nate the  need  for  the  sensing  vehicle  to  be  in  close  proximity 
to  a submarine.  The  conventional,  localized  method  of  detec- 
tion using  sonar  would  be  replaced  by  a more  wide-ranging 
system. 

Hierarchy  Construction 

The  first  step  in  the  analysis  was  to  develop  a set  of 
mutually  exclusive  and  exhaustive  hypotheses.  The  analyst 
felt  that  if  such  a program  existed  in  Country  B,  it  might 
take  one  of  two  possible  forms.  It  could  be  a pure  research 
program  aimed  at  obtaining  an  understanding  of  the  physical 
processes  involved,  or  it  could  be  a program  aimed  at  ob- 
serving and  measuring  possible  surface  effects.  The  third 
possibility  to  be  considered  is  that  there  is  no  program.  He 
therefore  developed  the  following  hypotheses: 

H. . There  ^ a research  program  to  investigate  surface 
^ effects  of  a submerged  submarine  for  potential 
military  use.  The  emphasis  is  on  obtaining  a 
thorough  understanding  of  the  physical  processes 
involved  in  the  generation  of  surface  effects. 

H,.  There  ^ a research  program  to  Investigate  surface 
^ effects  of  a submerged  submarine  for  potential 
military  use.  The  emphasis  is  on  observation  and 
measurement  of  the  surface  effects,  with  little 
attenticn  cjiv-ui  to  the  p'Uy ¥ i ra..  processes  involved. 

H..  There  is  no  program  to  investigate  surface  effects 
^ of  a submerged  submarine. 

The  analyst  next  developed  a list  of  six  RsD  program 
activities  which  would  be  likely  to  occur  if  the  hypothesised 
research  programs  existed.  These  activities  vrould  be  necessary 
to  produce  solutions  to  critical  technological  problems  and 
would  probably  produce  observable  data  indicating  their  presence. 
These  six  activities,  comprising  the  second  level  of  the 
hierarchy,  are: 

, Wake  Formation 

2 

A . Wake  Observables 

A^.  Ambient  Ocean  Conditions 

A^.  Infrared  Radiometry  of  the  Ocean  Surface 

A^*  Radar  Scattering  by  Surface  Waves 

A®.  Pattern  Distortion  by  Ocean  Waves 

From  his  own  knowledge  of  the  problem  and  with  the 
assistance  of  experts  in  the  technical  areas  encompassed  by 
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the  six  activities,  the  analyst  was  able  to  develop  a number 
of  indicators  which  would  be  diagnostic  about  the  existence 
of  the  R&D  activities.  These  the  analyst  related  to  the 
activities  in  the  following  manner: 

. Wake  Formation 

111  _ a?urbulent  Wake 

112  _ Fluid  Displacement 
ll-  “ Wake  Measurement 
ll4  _ Internal  Waves 

2 

A , Wake  Observation 
l21  - Modeling 
z22  _ Surface  Roughness 
l23  - Slick  Formation 
x24  - Temperature  Measurement 

A^.  Ancient  Conditions 

- Wave  Generation 
l32  - Heat  Transport 

x33  . Temperature  Fluctuations 

- Cloud  Reflections 
l35  - Surface  Slicks 

A^ . Infrared  Detection 

l41  - Detector  Development 
ly  - IR  Systems 
1^3  - Atmospheric  Effects 
l44  _ signal  Processing 

5 

A . Radar  Detection 

151  - Theoretical  Models 

152  _ Differential  Roughness 

- Radar  Measurements 

A^.  Pattern  Distortion 

161  _ Weather  Conditions 

162  _ wind  Conditions 

163  _ Wave  Conditions 

At  this  point  the  analyst  had  sufficiently  decomposed 
the  problem  so  that  he  could  lay  out  the  hierarchical  struc- 
ture shown  in  Figure  D-6,  which  linked  all  the  available 
data  to  the  top-level  hypotheses.  Note  that  two  activities, 
a2  and  A^,  are  not  in  this  diagram  (Figure  D-6),  indicating 
that  there  were  no  available  data  related  to  these  activities. 

It  is  evident  from  Figure  D-6  that  most  of  the  informa- 
tion in  the  observed  data  is  related  to  the  hypotheses  under 
analysis  indirectly  through  a chain  of  indicators  and  activ- 
ities. As  an  example,  datum  Dill  is  the  information  con- 
tent of  a paper  published  recently  by  Dr.  X,  This  datum  is 
diagnostic  in  discriminating  between  the  two  hypothesized 
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states  of  indicator  which  is  in  turn  diagnostic  between 

the  states  of  Ai,  and  therefore  diagnostic  among  the  hypotheses. 
It  is  important  to  note  that  the  impact  of  data,  linked  to  the 
hypotheses  through  the  intermediate  variables,  will  be  filtered 
by  each  level  through  which  they  pass.  For  this  eason  the 
intermediate  variables  must  be  defined  carefully,  and  the 
probabilities  linking  the  variables  must  be  assessed  as  accu- 
rately as  possible.  On  the  other  hand,  data  d7 , d^,  and 
are  linked  directly  to  the  hypotheses. 

Aasosaments  and  Mathematical  Solution 


The  analyst  first  assessed  the  prior  probabilities  of 
different  hypotheses  occurring.  He  felt  that  the  odds  were 
4:1  in  favor  of  H3,  "no  program."  He  also  felt  there  was  an 
equal  probability  that  the  programs  represented  by  H],  and  H2 
existed.  Thus  his  prior  probability  assessments  were  P(Hi)  « 
0.10,  P(H2)  - 0.10,  and  P(H3)  - 0.80. 

The  analyst  next  assessed  the  linkages  of  data  o'?,  D^, 
and  d9  to  the  hypotheses,  and  the  linkages  of  the  lower-level 
data  to  the  indicators  about  which  they  were  diagnostic.  He 
found  it  more  convenient  to  express  these  relationships  in 
terms  of  likelihood  ratios  as  follows: 


- Reference  to  Dr.  Y in  paper  by  Dr.  X.  The  analyst 
felt  that  this  datum  was  least  likely  given  H3. 

He  then  assessed  the  relative  likelihoods  of  the 
datum,  given  the  other  hypotheses  in  comparison  to 
H3- 

P(D^|Ht  5 

• — - - - - 1.2 

P(d'|H3  5 

P(D^lH-  8 

= — - » - » 1.14 

P(D  [Hj  7 


D 


8 


He  then  expressed  these  relationships  in  ratio  form: 
P(d'^IHj^)  :P(d’^|H2)  :P(d’^1H3) 

1.2  : 1.14  : 1 

Interest  by  Dr.  Y in  Country  C work  during  visit  to 
Country  C facilities: 

P(D®|Hj^)  :P(D®1H2)  :P{D®|H3) 

2:2:1 
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- Co- location  of  Dr.  X and  Dr.  Yt 
P(d91Hi) :P(d9!H2) ;P(d91H3) 

4:2:1 
Dili-  Content  of  paper  by  Dr.  X: 

P(d111| jll)  5 

P(Dlllllll)  1 

d131-  Existence  of  towed/  controllable  sensor  platform 
suitable  for  probing  submarine  wakes: 

P(d131|i13)  5 

P(d1311i13)  2 

d331_  piiper  describing  design  characteristics  of  an 
Ir  radiometer: 

P(d331|i33)  6 

P(d331|i33  1 

d441_  Paper  discussing  differential  radiometry: 
P(d4411i44)  20 

■iii«  -■■■  m 

P(d4411i44)  7 

d661-  Paper  by  Dr.  X describing  effects  of  turbulent 
seas  on  oil  platform  operations: 

P(d661|iJ1)  2 

P(d6611i|1)  3 

d631-  Paper  by  Dr.  X discussing  turbulent  wake  of  a 
fluid  in  flux: 

P(d631|iJ3)  7 

P(D®311i63)  5 

Next/  the  analyst  needed  to  assess  the  conditional 
probabilities  relating  the  indicators  lH  and  ll3  to  activity 
a1,  and  the  activity  to  the  hypotheses.  After  making  these 
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judgments,  he  set  out  assessments  pertaining  to  the  branch 
of  the  hierarchy  in  the  form  of  a deductive  structure, 

Figure  D-7A. 


A.  DEDUCTIVE  STRUCTURE 


B.  INDUCTIVE  STRUCTURE 
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Figure  D-7 

HIERARCHICAL  STRUCTURE  FOR  BRANCH  A^ 


The  results  of  working  up  the  hierarchy  from  the  obser 
vation  of  data  and  d131  are  given  in  Figure  D-7B. 

Calculations  are  shown  in  the  steps  below. 


p(d^^^IaJ) 


- (5)  (0.95)  + (1)  (0.05) 
« 4.75  + 0.05  - 4.80 


P(D^^^]  iJ^)P(I£^|A2)  + P(D^^^|l2  )P(l2 


(5)  (0.04)  + (1)  (0.96) 
0.20  + 0.96  « 1.16 


^^^IaJ)  = p(d^^^1iJ;^)p(iJ^1aJ)  + p(d^^^1i2^)p(I2^1a];) 

- (5) (0.90)  + (2) (0.10) 

- 4.50  + 0.20  - 4.70 

^^^|A^)  - P(D^^^llJ^)P(l“|A^)  + P(D^^^|iJ^)P(I2^1aJ) 

- (5)  (0.50)  + (2)  (0.50) 

» 2.50  + 1.00  - 3.50 


- P(D^^^1aJ)P(D^^^1aJ) 

» (4.80)  (4.70)  - 22.6 

- P(D^^^1aJ)P(D^^^1A2) 

» (1.16) (3.50)  - 4.06 


» P(D^^^D^^^1aJ)P(aJ|Hj^)  + P(D'^^-^D'*''’'^1  aJ: 

- (22.6)  (0.95)  + (4.06)  (0.05) 

- 21.5  + .20  - 21.7 

- P(D^^S^^^lAj;)P(AjlH2)  + P(D^^^D^^^|A2: 

- (22.6)  (0.35)  + (4.06)  (0.65) 

» 7.92  + 2.64  « 10.6 


lll„131ia. 


IH2) 


IH3)  - P(D^^^D^^^1aJ)P(aJ1K3)  + P(D^^^D^^^!a^ 

- (22.6)  (0.03)  + (4.06)  (0.95) 


1.13  + 3.86  - 4.99 


The  analyst  next  assessed  the  necessary  conditional 
probabilities  relating  the  indicators  to  the  activity  A^, 
and  the  activity  to  the  hypotheses,  as  shown  in  Figure 
D-8A. 


A.  DEDUCTIVE  STRUCTURE 


B.  INDUCTIVE  STRUCTURE 
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Figure  D-8 

HIERARCHICAL  STRUCTURES  FOR  BRANCH  A^ 


He  then  perfomed  the  calculations,  as  shown  below, 
to  obtain  the  values  for  the  inductive  structure,  as  shown 
in  Figure  D-8B. 

P(D^^^IaJ)  - (6)  (0.95)  + (1)(0,05) 

- 5.70  + 0.05  - 5.75 


P(D^^^1A2) 


(6) (0,50)  + (1)  (0.50) 
3.00  + 0.50  - 3.50 


P(D^^^IHj^) 


(5.75)  (0.95)  + (3.50)  (0.05) 
5.46  + 0.18  « 5.64 
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P(D‘''"-|H2)  = (5.75)  (0.95)  + (3.50)  (0.05) 
■ 5.46  + 0.18  • 5.64 

P(D^^^1H3)  - (5.75)  (0.70)  + (3.50)  (0.30) 
- 4.03  + 1.05  - 5.08 


Figures  D-9  and  D-10  show  the  deductive  structures 
assessed  by  the  analyst  for  branches  and  and  the 
results  of  solving  the  inductive  structure. 
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Figure  D-9 

HIERARCHICAL  STRUCTURES  FOR  BRANCH  A^ 


A.  DEDUCTIVE  STRUCTURE 
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Figure  D-10 

HIERARCHICAL  STRUCTURE  FOR  BRANCH  A® 


Figure  D-11  shows  the  final  step  in  solving  the  inference; 
it  uses  the  computed  likelihoods  for  the  activity  branches 
and  the  given  likelihoods  for  the  data  impacting  directly  on 
the  hypotheses.  Each  likelihood  has  been  normalised  so  that 
its  impact  on  the  final  likelihood  is  easily  visualized. 
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INDUCTIVE  STRUCTURE  FOR 
FINAL  STEP  IN  INFERENCE  SOLUTION 

The  final  step  in  the  calculations  is  as  follows: 


(4.35) (1.11) (1.20)  (1) (1.20) (2)  (4)  - 55.62  . 


Similarly! 


P(d1H2)  » (2.12)  (1.11)  (1.39)  (1.07)  (1.14)  (2)  (2)  - 15.96 
P(D|H3)  - (1) (1)  (1) (1.06)  (1) (1) (1)  - 1.06  . 

Normalizing  these  gives  li]celihoods  of  52.5,  15.1  and  1. 
Using  Bayes's  Theorem, 

P(H^|D)  52.5  0.1  6.56 

P(H2lD)  - 15.1  X 0.1  - 1.89 

.:^H3lD)  1 0.8  1 


And  from  this  the  posterloi:  probabilities  are 


P(HilD)  - - 0.694, 

P(H2|D)  • - 0.200, 

PCHgiD)  - 5^5  - 0.106. 
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